AVA annual report guidance notes (2006-07)

(report due at QAA by 3 December 2007)

Part 1 : Annual report guidance to AVAs 2006-07

Introduction
Preparation and presentation
Submission
Annual report checklist
Sections for inclusion in annual reports
Standard appendices
The 2007 special appendix

Introduction

Purpose

The consideration of access validating agency (AVA) annual reports and statistics plays a key role in the monitoring of AVAs by the Quality Assurance Agency for Higher Education (QAA). Together with the system of AVA review, annual reports provide the means by which QAA can satisfy itself and assure others that an AVA is fulfilling the terms of its licence and continues to be a fit body for the validation of Access to Higher Education (HE) programmes and the award of Access to HE qualifications.

The preparation of annual reports involves AVAs in a process of review and evaluation of their work. This contributes to AVAs' own quality assurance and enhancement activities, and provides a means by which AVAs can inform others about their activities and developments.

Main emphases

The AVA annual report brings together the outcomes of the AVA’s monitoring procedures on quality and standards, and reports on activities aimed at developing and enhancing Access to HE. The report’s appendices include copies of formal plans, minutes and other records, providing an evidential base for the comments, analysis and judgements in the main text of the report. The process of self-evaluation involved in the preparation of the report requires AVAs to make an assessment of the effectiveness of their performance against QAA’s published AVA licensing criteria in relation to a range of matters including governance, management, academic standards, quality assurance, development and enhancement.

Scrutiny and outcomes

Annual reports are submitted to the Access Recognition and Licensing Committee (ARLC), which is charged with considering whether QAA can, on the basis of the report provided, be confident that each AVA is fulfilling the terms of its AVA licence.

Following the ARLC’s annual report scrutiny process, AVAs receive individual feedback on their reports and are notified of whether their reports have been:

a          approved
b          accepted (with a request for further clarification or information)
c          not approved (in which case further action is taken).

If the ARLC’s scrutiny of the report and any follow-up suggests that there is some doubt about whether the AVA is meeting the terms of its licence, there may be a request for further information or a special meeting or review may be arranged. If the ARLC’s concerns are confirmed by this further investigation, it may require the AVA to take certain actions as a condition of continuation of the licence.

Annual reports also make an important contribution to the AVA review process, forming part of the source material on which review teams make their assessment of AVAs. The ARLC may ask review teams to follow up queries raised by annual reports, and AVAs are asked to provide a response to recommendations resulting from review findings in their subsequent annual report. AVA annual reports thereby have a central role of this quality monitoring circle.

Preparation and presentation

The AVA annual report reflects on Access to HE-related activities during the year which is the subject of the report, from the perspective of the following year. It looks back to the achievements and challenges of the previous year and reviews the AVA’s activities and developments, adopting an evaluative approach, which seeks to be objective and analytical, rather than merely descriptive or celebratory. As well as noting successes and positive outcomes, the report also considers problems or difficulties and discusses how these affected the AVA’s work during the report year and how the AVA has responded.

There are standard requirements for all annual reports, but AVAs may also include additional information within the section headings provided, if it helps to give a more complete account of their activities. AVAs are encouraged to emphasise the particularity of their activities and of the Access to HE provision for which they hold awarding body responsibilities.

In preparing the report, AVAs should ensure that:

full account is taken of feedback provided by QAA to the AVA in previous years
material is organised according to the sections given in this guidance (see Sections for inclusion, page 6), with the headings listed here used as section headings for the report. Where additional material is included, or material could be included in more than one section, AVAs apply their judgement about where it is best placed for clarity and in order to avoid repetition
a tone and style which is appropriate to the report’s formal status is used throughout
acronyms, local terminology and references to local circumstances are explained in sufficient detail to allow readers unfamiliar with the AVA to understand its work
lengthy, detailed narrative is avoided: the main text (excluding appendices) is around 5,000-7,000 words, but use is made of sub-headings, lists, tables, charts and process diagrams, if it helps to provide information more succinctly and directly
full analysis is provided where requested, particularly in relation to the AVA’s data
appendices are used to support and verify statements in the main text: appendices are clearly cross-referenced to the main text of the report and material is not included in appendices without some reference to its significance in the main text of the report
double-sided copying is considered for lengthy documents, in order to reduce the overall bulk of reports
the report is securely bound, and presented as a single document under one set of durable covers which will withstand several readings and journeys.

Before submitting the report, AVAs should check each copy of the report and ensure that:

the report has been proof-read and any corrections have been made to each copy
all specified material has been included, including the required appendices (see the Annual report checklist, page 5)
all sections of the main text of the AVA’s report use the same titles as in this guidance
appendices appear together at the end of the report and the individual appendices are clearly separated from each other and from the main text of the report
appendices are clearly numbered and titled (using the same title on the document as on the contents page and in any references made in the main text)
cross-referencing is accurate
page numbering is used throughout and is consistent with the numbers given in the contents page
pages have been properly collated.

Submission

AVAs should submit one electronic copy and five printed copies of the annual report, to arrive at QAA’s offices no later than Monday 3 December 2007.

Electronic copies of reports should be sent by email attachment to access@qaa.ac.uk. AVAs should also send a separate email to the same address, stating that the report has been sent. This will alert QAA to any difficulties that might arise with the electronic delivery of the report.

Printed copies of reports should be sent to:

Ann-Marie Karadia
Project Officer (Access to HE)
The Quality Assurance Agency for Higher Education
Southgate House
Southgate Street
Gloucester
GL1 1UB

Back to top

Annual report checklist

Statement of formal approval (signed and dated)
Contents page (including sections of main text and appendices)
Main text	Appendices
1 Introduction	Annual report preparation process diagram (including the process of formal approval).
2 Legal, constitutional and governance matters	Diagram of the AVA’s committee structure, with reporting lines. List of the AVA committees (include membership/representation and names/affiliation of members during the report year). Dates of committee meetings during the report year (any cancelled or dates changed?). Committee minutes (including finance committee). List of AVA stakeholders (marked to show HEIs and Access to HE providers), with additions and deletions since the previous year.
3 Strategic, management and operational matters	Strategic plan and targets for report year. Strategic plan and targets for the coming year. Staff structure (eg organisational diagram) and list of current staffing (including job titles; fractional and temporary posts; dates of new appointments). Externally verified accounts.
4 Quality assurance, enhancement and development	Samples of new quality assurance documentation. AVA evaluations of providers’ programme reports. List of the AVA’s development events and target audiences.
5 Academic standards	AVA evaluations of moderators’ reports. Moderators’ names, affiliation and length of service.
6 Commentary on statistical data: programme data student data	Access to HE providers and programmes, with additions and deletions since last year. A print-out of 2006-07 statistics submission. Previous two years’ statistics submissions.
Response to AVA review recommendations (where an AVA review has taken place since the last annual report was prepared)
Conclusions
		Special appendix

Sections for inclusion in AVA annual reports

Evidence of approval

The final version of the report must be formally approved by the AVA’s governing body. The ARLC will not approve a report that has not been approved by an AVA’s governing body. This approval should be evidenced through a statement which specifies the date of the meeting at which the report was approved, and is signed by the Chair of the governing body. Please provide this evidence of the report’s approval on a separate sheet at the front of the report.

Contents page

Section headings with associated page numbers should be given. Appendices should also be clearly numbered and listed on the contents page.

Introduction

This section should provide an overview of the AVA's activities and achievements, noting major changes and developments in the environment for the AVA’s work. Include, in particular:

contextual information to provide basic introduction for readers new to the AVA (eg the organisation’s size and structure; geographical reach and scale of its activities; relationship between Access to HE and any other areas of activity)
the AVA’s position at the beginning of the year, its priorities for Access to HE, and the major challenges and opportunities it faced
internal and external factors influencing the organisation's progress in relation to Access to HE during the report year
process for the preparation of the annual report, which may be shown in a diagram.

1 Legal, constitutional and governance matters

This section should describe any significant changes in legal or constitutional status and assess the effectiveness of the AVA’s governance structures. Include, in particular:

legal, constitutional or governance changes and reasons for them
changes to the involvement of stakeholders (including receiving HE institutions and Access to HE providers), including any known reasons for changes
committees’ activity (in terms of the range of business undertaken, frequency of meetings and patterns of attendance)
policy decisions or measures taken or planned by the AVA in relation to developing the scope and range of its activity
other key decisions taken affecting the AVA’s work and how they arose.

The text of section 1 should be cross-referenced to relevant appendices, including:

a list of stakeholders, marked to show HE institutions and Access to HE providers,
and indicating any additions and deletions since the previous year
a diagram of the AVA’s committee structure, showing reporting lines
a list of the AVA’s committees and indication of their composition in terms of representation and/or named individuals and their institutional/organisational affiliation
a list of dates of committee meetings which took place during the report year, including any cancelled meetings or changes to dates
minutes of all meetings of key AVA committees, including the AVA’s governing body and the committee that has responsibility for finance, and for quality assurance of AVA business. If, for reasons of confidentiality, it is necessary to submit edited minutes, the points at which any material has been omitted must be made clear.

2 Strategic, management and operational matters

This section describes strategic and policy developments; it evaluates the success of the AVA’s plans in developing Access to HE; and assesses the impact and effectiveness of changes to the AVA’s management and operational functions.

Include, in particular:

strategic planning - summary of the AVA’s approach, including the process for approval of the AVA’s strategic plan and for monitoring targets. A new strategic plan will not necessarily be devised on an annual basis, but there should be an indication of when, and by whom, the plan is reviewed and targets updated. The format of strategic plans varies, but all AVAs’ plans should include - or make reference to a separate document (for example, a ‘business plan’ or ‘action plan’) which includes - formally approved targets for the development of the AVA and its Access to HE work. The status of any plans, their relationship to each other, and the period to which they relate are clearly stated. Tabulated plans (included either in the main text or the appendix) relate strategic objectives to specific targets which are measurable and time-limited and which indicate designated individuals/groups with responsibility for meeting the targets
response to the report year’s targets - review of the AVA’s progress in meeting targets it set for the year, including comment on each target, with an indication of whether, and how, it was achieved. Where an action has not been achieved, the report should explain why it has not, and how the AVA intends to pursue it. Any unmet target should be shown as redundant or revised to respond to changes; or deferred and transferred to the coming year’s action plan with a new target date
targets for the coming year - the plan for the coming year, with new or updated targets: targets for Access to HE should be related to the AVA’s broader strategic aims, and to areas identified elsewhere in the report (or in the previous year’s report) as requiring action
resources - assessment of the organisation’s financial position and changes to financial management; contribution and costs of AVA work to the organisation’s finances; changes to accommodation and its adequacy for the AVA’s functions; staffing (including new posts with, or re-allocation of, Access to HE responsibilities)
communications - overview and developments; evaluation of success of communications and promotional work with Access to HE providers and others
self-assessment and risk assessment - summary of outcomes of internal processes undertaken by the AVA during the report year.

The text of section 2 should be cross-referenced to relevant appendices, including:

strategic plan and targets for the report year
strategic plan and targets for the coming year
staffing structure and staff list, with job titles, and indicating fractional/temporary posts, and dates of new appointments.
full audited accounts for the most recent complete financial year. If this presents any difficulties, explain these in the main report. Please ensure that the accounts have been signed and dated.

The following common circumstances should be dealt with as indicated:

Problem: the AVA’s financial year is not the same as the reporting year.
Response: submit the audited accounts for the most recent financial year (indicating the period that it covers), and include management accounts for the period of the reporting year that is not covered by the audited accounts.
Problem: the AVA is an unincorporated association and its accounts are not audited.
Response: submit detailed accounts that have been externally verified by some other means, and explain how they have been verified.
Problem: The AVA’s accounts are audited through a ‘host institution’.
Response: Submit accounts that relate to the AVA’s activities, extracted from those of the ‘host institution’, if they are separately identifiable. Also submit the management accounts which are presented to the AVA’s governing body. If the AVA’s accounts are not separately identifiable within the host institution’s accounts, explain in the main text how the AVA’s accounts are verified, and submit the management accounts which are presented to the AVA’s governing body.

3 Quality assurance, enhancement and development

This section records developments in quality assurance processes relating to Access to HE; the reason for their introduction and what improvements they are intended to bring about in the quality of Access to HE programmes or the AVA’s assurance of their quality; and an assessment of their effectiveness since being introduced, where possible.

The section also assesses the AVA’s success in establishing its position as a focus of development, through enhancing current Access to HE provision, initiating new developments, and engaging with developments initiated by others. Include, in particular:

changes to the AVA’s quality assurance processes for course development; validation; revalidation; and course monitoring, with explanation for, and evaluation of, any changes
conclusions drawn about the quality of Access to HE provision from its processes for course monitoring, including the outcome of its evaluation of programme (or centre) reports, highlighting positive developments; general trends and patterns; and particular areas of difficulty, and areas for further development
summary and evaluation of development and enhancement events organised during the year, noting any particular outcomes
description and evaluation of the AVA’s involvement in regional or national projects,
as well as smaller-scale local initiatives, designed to improve or develop Access to HE; evidence of productive cooperation outside the AVA with other organisations involved
in widening participation in higher education and assessment of progress made in
these areas.

The text of section 3 should be cross-referenced to relevant appendices, including:

samples of new forms designed for quality assurance purposes (evidence of minor amendments, or forms used for other purposes would not be expected)
summary evaluations of programme (or centre) reports, including key findings
a list of development or enhancement events organised by the AVA (with dates) and their target audience.

4 Academic standards

This section reports on the outcome of the AVA’s procedures for monitoring academic standards on Access to HE programmes, and measures taken to assure their consistency and sufficiency. It also considers, and assesses the impact of, activities initiated by the AVA relating to assessment, standardisation, moderation and the award and issue of Access to HE certificates (or Diplomas). Include, in particular:

assessment of moderation process, particularly in relation to the impact of any recently introduced refinements or changes to the AVA’s procedures
outcomes of measures undertaken to monitor moderators’ performance
analytical response to outcomes of moderation activity through the AVA’s evaluation of moderators’/examiners’ reports. As well as recording successes and notable achievements, the response will note any apparent weaknesses or areas of concern, and the AVA’s intended actions in relation to these. AVAs are not expected to make judgements about named, individual programmes’ performance in their reports but the report mentions how the AVA has addressed unsatisfactory quality or standards it has identified
trends relating to the quality and standards of Access to HE provision and student achievement, as noted by moderators in their reports
any necessary explanation, especially where references are made to different categories of moderation or moderators which are not self-explanatory.

The text of section 4 should be cross-referenced to relevant appendices, including:

summary evaluations of moderators’/examiners’ reports, including key findings
moderators’ names, affiliation and length of service.

5 Commentary on statistical data

This section provides an analytical and in-depth commentary on the statistical data collected for the AVA’s statistical report to QAA (see Part 2 - Data submission - guidance to AVAs 2006-07), and provides the basis for a detailed discussion of how it relates to characteristics of Access to HE provision during the report year. The commentary provides the outcome of a process of analysis and interrogation of the data, identifying any anomalies and attempting to explain their cause: it attempts to explain, rather than describe, the data.

Where the statistics are self-evident or unremarkable, commentary may not be necessary, but the report should comment on the most striking features of the data and their significance and consider, in general:

local circumstances which help to interpret or explain particular features of the data, where this is known (supporting anecdotal evidence can be illuminating: mere conjecture is largely unhelpful)
features which may not be evident or obvious from the raw data included in the report alone, particularly any factors that may have distorted the aggregated statistics presented
differences in, and ranges of, statistical data received from different providers and whether there are reasons to explain these. Although the report itself will probably not discuss the performance of individual providers by name, reference in general terms to considerations of differential performance may be needed for an external reader to have a proper understanding of aggregated or averaged statistics received from a number of different institutions and programmes
reasons for incomplete data in any section, with details of action taken by the AVA to gain full data from all providers outlined
comparisons with previous years’ statistics, perceived trends and any known reasons
for these
comparisons between the AVA’s own data and published national data, with any known reasons for significant variance from standard patterns. Comments may helpfully also include references to statistical data gathered for other purposes, relating, for example, to comparative regional data.

The report will also include comment on, in particular:

programme data - changes to the provision for which the AVA is responsible, and whether this has changed the overall profile of Access to HE provision. Areas to note may include areas of expansion and contraction; changes to providers; types of programmes recognised; progress with validations/revalidations; programmes withdrawn; and gaps in provision. Information about recruitment to different types of programme (whether described by mode, location, explicit targeting or defined progression route) is also useful
learner data - comment on what is demonstrated by learner profiles about the extent to which provision is reaching its target groups. Learner achievement and progression should receive comment, particularly with reference to emerging trends. Although the statistical returns do not require information on reasons for student withdrawal, information gleaned from providers about trends or correlations with particular target groups may prove useful.

The text of section 5 should be cross-referenced to relevant appendices, including:

a list of QAA-recognised Access to HE programmes and providers, with an indication of additions and deletions since the previous year
a printed copy of the statistics submitted electronically to QAA
previous years’ statistics for at least the last two years, sufficient to demonstrate trends described in the commentary.

Response to recommendations set at AVA review
Only those AVAs which have recently been reviewed will include comment in this section. Information is provided about the way in which the AVA has responded to each recommendation. If the AVA has decided not to take action in response to any of the particular recommendations, the reason for this decision will be clearly stated. If the AVA’s response to review recommendations has been included in the AVA’s action plan, reference to each particular recommendation will be explicit and easily identifiable.

6 Conclusions

This section provides an opportunity for the AVA to provide a brief statement on how the events of the year have affected the organisation as a whole, its position at the end of the year, and the major challenges and opportunities and key priorities which it faces in the coming year.

Standard appendices

For a list of the required appendices, see the Annual report checklist (page 5).

Information in appendices supports the commentary in each section, as indicated under each of the section descriptions given in the previous section. It is expected that most appendices, with the exception of the 2007 special appendix (see below), will be documents which have been produced prior to the preparation of the annual report for the AVA’s internal purposes.

In instances where an appendix is not available in electronic format, the contents page should state that the appendix is available in hard copy only.

The 2007 special appendix

The 2007 special appendix will provide more detailed information about AVAs’ Access to HE-related finances than is required as a standard part of the annual report under section 2, although it may make reference to that section’s appendices. However, the detail sought may require a style of presentation of information, which is not the same as the AVA’s standard approach to financial reporting, and AVAs are likely to have to undertake some analysis of their own financial records in order to provide a satisfactory special appendix.

Where AVAs operate an integrated budgeting model for Access to HE and other business,
they should show Access to HE disaggregated from other sources of income and expenditure.
It is recognised that there may be some areas in which it is not possible to provide an absolute distinction between income or expenditure which is specifically allocated to or derived from Access to HE (for example, membership fees) and, in these areas, AVAs are expected to make a reasoned estimate about the proportion relating to Access to HE. Any estimates should be indicated as such and the basis for the estimate made clear.

The 2007 special appendix will explain, in particular:

how much of the AVA’s income in 2006-07 (in absolute and proportionate terms) was derived from its Access to HE activity, and the different sources of that income (for example membership charges, validation/revalidation fees, certification fees, consultancy) - and whether there was anything exceptional about income in 2006-07
how much of the AVA’s expenditure in 2006-07 (in absolute and proportionate terms) was allocated to Access to HE-related activity (for example staffing, administration, training, moderation, staff development, special projects) - and whether there was anything exceptional about expenditure in 2006-07.

The special appendix will also explain how the AVA uses financial information to plan and monitor its activities for Access to HE, including detail about the following:

the AVA’s process for financial planning, as it relates to determining priorities, planning activities, setting targets, setting budgets and risk management
how financial projections are made
how budgets are monitored, and who is involved in these processes
where responsibility lies within the AVA for different aspects of financial management.