The new MFHEA Framework makes EQA outcomes more predictable, but transparency remains incomplete on judgments, indicators, proportionality, follow-up, appeal, and pathways to Level 8 and advanced institutional statuses.

The consultation launched by MFHEA on the new Criteria for Outcomes & Follow-up Procedures Framework, open until 11 September 2026, is understandably attracting attention among providers affected by External Quality Assurance processes.

In recent days, Malta Quality Education has received requests from several Higher Education providers for clarification and technical evaluation on the possible implications of the new Framework. We therefore considered it useful to publicly share some of the main issues that emerged from our analysis, with the aim not of suggesting to providers what position to take, but of contributing to a more informed reading of the document so that each institution can independently formulate its own observations in the context of the consultation.

The document under consultation not only introduces a new way of representing the results of External Quality Assurance: it tries to define, more explicitly, but the link between the judgment also attributed to the individual Standards, the level of institutional risk, the decision on accreditation, any regulatory measures and the subsequent follow-up.

For Higher Education providers, it is therefore a substantial consultation, because the new model can directly affect the renewal of the license, the possibility of welcoming new students, the expansion of programs and, for institutions aiming for more advanced statuses, the future regulatory progression.

The declared purpose of greater clarity responds to a real criticality.

The ENQA/EQAR dossier had in fact asked MFHEA to make the criteria for outcomes of the various accreditation procedures more explicit, public and consistently applied. The new Framework addresses this problem mainly downstream, making the consequences of a Full, Substantial, Partial or Non-Compliant judgment more predictable.

But it is precisely this greater predictability that forcefully raises the question that should be at the heart of the consultation: with what methodology does the Panel arrive, in the first place, to assign one of those four judgments to the Standard?

Mandatory indicators and graded judgments: the central contradiction

The point is deeper than simply the absence of a mathematical formula. The MFHEA manuals of providers and programme accreditations establish that the Standards are composed of Minimal Indicators and, when applicable, Performance Indicators.

Minimal Indicators are described as a mandatory level of achievement and must be met for accreditation purposes and maintained in subsequent audits; Performance Indicators also become mandatory, according to the applicable procedure, starting from the first cyclical review or renewal. The new Outcomes Framework confirms this architecture, indicating that Minimal Indicators apply for Initial Provider Accreditation and that Performance Indicators also apply for subsequent procedures.

This is where a contradiction arises that the document under consultation does not address. If all applicable indicators are mandatory levels to be respected, what space is left to declare a Substantially Compliant Standard when one or more mandatory indicators are not fully met? When does the failure to satisfy a mandatory indicator produce Partial Compliance and when does it determine non-compliance? And, if an unsatisfied indicator can be absorbed within an overall positive judgment, in what sense does that requirement continue to be mandatory?

The two logics are not necessarily incompatible, but they must be reconciled.

The indicators can be understood as individually mandatory requirements, the non-satisfaction of which generates a defined consequence, or as mandatory evaluation elements that the Panel weights in the overall framework of the Standard. The current system seems to use the language of the first approach and, at the same time, an overall evaluation methodology compatible with the second, without explaining in a sufficiently transparent way how the two logics should coexist. This is, in our opinion, the central methodological criticality that the new Framework should have addressed before linking ratings to much more incisive regulatory consequences.

The lack of transparency between indicators and judgement

This apparent contradiction could be resolved through a published judgement methodology, capable of clarifying the operational meaning of the term mandatory and regulating the transition from indicators to the overall judgment of the Standard.

It is precisely this methodology that does not emerge today in a sufficiently determined way from the documents examined. The four categories of compliance are defined, but the definitions remain general: Substantial indicates broad alignment with the Standard; Partial indicates that some parts are satisfied and others are not, with an implementation that is not effective enough; non-compliant occurs when the institution does not comply with the Standard. These formulations describe the result, but do not establish an operational rubric that makes the path by which the Panel arrives at that result verifiable.

If a Standard contains numerous Minimal and Performance Indicators, what weight does each one assume? Is a single Minimal Indicator not respected enough to produce Partial Compliance? What if it concerns student protection, assessment integrity or governance? Can several Performance Indicators that are not fully satisfied have a lower weight than a single particularly material Minimal Indicator? When does a deficiency stop being circumscribed and become systemic? The Panel must necessarily retain professional judgement, but transparency requires that that technical discretion be channeled by knowable, consistent and justifiable criteria.

The “missing middle” of the new Outcomes Framework

The new Framework is much more precise in regulating what happens after the rating has been assigned. A given combination of Fully, Substantially, Partially and Non-Compliant produces a level of risk and a regulatory consequence. On the other hand, the previous segment remains much less determined: how the evidence collected on the individual indicators produces the overall rating of the Standard. It is the true missing middle of the new architecture.

The sequence can be represented in simple terms: indicators and evidence, professional judgement of the Panel, rating of the Standard, level of risk, decision and follow-up. The document under consultation makes the last steps more predictable, but the moment in which the rating is born remains entrusted to general definitions. It is not a question of asking for an algorithm that replaces the experts. It is a question of asking that professional judgement become structured professional judgement: an assessment in which the weight of materiality, the severity of the deficiency, the systemic nature, the impact on students, recidivism and the concrete ability to remediation are made explicit.

Proportionality: before counting the Partials, it is necessary to understand how they are born

Table 2 dedicated to the renewal of accreditation makes the problem clear. One, two or three Partial Compliances, in the absence of non-compliant standards, lead to Moderate Risk; four Partial Compliances determine High Risk; a single non-compliance also leads to High Risk. This level can be followed by restricted licences, blocking of new enrolments, impossibility of expanding programmes and institutions and specific measures to protect students. If the subsequent follow-up does not lead to at least Substantial Compliance for the Standards concerned, the Framework provides for the revocation of the license.

The numerical threshold is clear, but the proportionality of the system depends on the quality and transparency of the judgment that feeds that threshold. Four Partials referring to limited, documentary, or quickly remediable deficiencies can formally produce the same level of risk generated by a single non-compliance that affects an essential aspect of governance, assessment or student protection. Before even discussing whether “four Partial = High” is a correct threshold, it is therefore necessary to clarify why each of those Standards has been qualified as Partial and not Substantial, or why a given finding has produced non-compliance instead of Partial.

The consultation specifically asks whether the thresholds are fair and proportionate. A technically based answer should therefore question not only the number of Standards involved, but also the methodology that leads to their classification. Without a transparent criterion of materiality and severity, the count may appear uniform while the data that is counted remains highly dependent on the discretion of the individual Panel.

University and Self-Accreditation: when a single rating can decide the status

In Tables 3 and 4 the theme becomes even more delicate. For University Status and Self-Accrediting Status, a single Partial Compliance or a single non-compliance can lead to High Risk and the rejection of the status or its renewal. The distance between Substantially Compliant and Partially Compliant therefore ceases to be a technical nuance and can become decisive for the institutional future of the provider.

The new Standard of Evidence usefully increases the visibility of expectations towards institutions aspiring to advanced status but knowing what evidence to present is not the same as knowing what level of evidence will be considered sufficient to obtain Full, Substantial or Partial Compliance. The more relevant the consequences of a single rating become, the more robust the rating descriptors, the evidential sufficiency methodology and the calibration mechanisms between different panels must be. Transparency, in this context, does not only concern the provider: it also concerns the Authority’s ability to demonstrate that comparable cases are evaluated through comparable criteria.

ESG 2.5: public criteria, not just public categories

This point is directly linked to ESG 2.5. The Standard requires that outcomes and judgements deriving from External Quality Assurance be based on explicit and published criteria, applied consistently; the Guidelines add that these criteria must be pre-defined, consistently interpreted and evidence based. Publishing the four categories of compliance is necessary, but it does not completely solve the problem if the criterion that concretely separates one category from the other remains implicit.

ENQA had already identified the criticality. In 2024, it recommended that, for each accreditation procedure, the criteria used to reach the final decision be clearly established and that all EQA criteria of outcomes be published and communicated. EQAR was even more explicit, noting that there were not sufficient guiding criteria to guarantee consistent judgement in the various provider and programme accreditation procedures and that there was a lack of a systemic approach to ensuring consistency in decision making.

MFHEA stated, in discussion with EQAR, that it is developing guidelines for the Quality Assurance Committee and peer reviewers to refine the criteria for outcomes. The document under consultation today is clearly a response to that need, but it seems to focus above all on the relationship between rating, risk level and regulatory outcome. If the previous step, indicators to rating, is not regulated with equal transparency, the response to the ENQA/EQAR recommendations risks remaining incomplete at the very point where the judgement is formed.

ESG 2.6: the judgment must be reconstructable

ESG 2.6 requires expert reports to contain evidence, analysis and findings, conclusions and recommendations. It does not mean turning a peer review into a judicial decision, nor depriving the Panel of its academic and professional expertise. It means, however, that the conclusion should be rationally reconstructable from the evidentiary basis and analysis that precede it.

It should be possible for the provider to understand not only which indicators were deemed insufficient, but why their number, nature and relevance led precisely to Partial and not to Substantial, or to Non-Compliant and not to Partial. A detailed description of the findings is important, but it is not a substitute for a published judgement methodology. Substantial transparency requires that the final evaluation step be motivated with respect to knowable criteria and not only entrusted to the professional conclusion of the Panel.

The appeal exists, but it must be concretely exercisable

This issue takes on further relevance considering ESG 2.7. The Guidelines provide that an institution can challenge an outcome when it demonstrates that the decision is not based on sound evidence, that the criteria have not been correctly applied or that the process has not been consistently implemented. The same approach appears in the appeal system described by MFHEA in its Self-Assessment Report: the grounds of appeal include the incorrect application of the criteria and the absence of sound evidence and arguments to support the decision.

Here a logical difficulty emerges. To argue that “the criteria have not been correctly applied”, the provider must be able to know with sufficient precision which criteria govern the choice between Substantial, Partial and Non-Compliant. If the published parameter remains general, the provider will be able to contest a fact, a document or the reading of a single indicator, but will have more difficulty in contesting the leap in evaluation that transforms those findings into the rating of the Standard. The appeal remains formally available, but its effectiveness depends on the transparency of the benchmark used to form the judgement.

Follow-up, recommendations and risk of cumulative effects

Follow-up is presented by the Framework as a tool for continuous quality enhancement, and it is correct that an EQA system allows institutions to correct weaknesses and demonstrate improvement. However, the document also states that unmet recommendations will be considered severe violations under Regulation 14(1)(c) of S.L. 607.03, with possible recourse to the measures provided for by the legislation. Such a broad wording requires clarification, because not all recommendations necessarily have the same nature or materiality.

The Programme Accreditation Standards and Procedures itself distinguishes between Mandatory Recommendations, Key Recommendations and Recommendations for improvement. If the system already recognises different categories of recommendations, proportionality and transparency require that any failure to comply with them should also be assessed in relation to the function and seriousness of the original recommendation. Otherwise, a finding born as an enhancement element could acquire, through subsequent steps, a much stricter regulatory value without the Framework making the escalation criterion sufficiently visible.

Remediation recognized for licensing, but not always for progression

The topic of follow-up becomes even more complex when the Framework is read together with the three Higher Education policies published by MFHEA on August 21, 2026. The policies relating to progression to MQF Level 8, Self-Accrediting Status and University Status provide, for specific eligibility requirements, that outcomes achieved following a follow-up procedure are not taken into consideration.

This creates a systemic tension: remediation may be considered sufficient to re-establish or confirm a normal licensing position, but the EQA cycle itself may remain unusable for the purposes of future regulatory progression. For a provider planning Level 8, Self-Accrediting or University Status, a successfully corrected Partial Compliance can therefore produce effects for years. The transparency of the new system should also make this temporal effect visible and clarify why a recognized remediation for the license should not have the same value in the regulatory track record.

Standard 10 and Level 8: a sequence still to be clarified

The Policy on the Progressive Delivery of Qualifications at MQF Level 8 requires, among the elements of institutional maturity, at least one provider accreditation renewal with Full or Substantial Compliance on all Standards, including Standard 10. Annex 1 of the new Outcomes Framework, however, indicates Standard 10 as applicable to universities and institutions that already deliver Level 8 programmes.

The combined reading produces a practical question: how can a provider not yet authorized to provide Level 8 demonstrate a previous positive outcome on a Standard described as applicable to those who already provide Level 8? It is possible that MFHEA intends a prospective assessment of research capacity, but if this is the interpretation it should be made explicit. Here too, the transparency of the regulatory pathway is essential to prevent the requirement from appearing circular or being interpreted differently in individual cases.

Self-Accrediting: track record, exceptional route and route consistency

A similar problem concerns Self-Accrediting Status. The Standard of Evidence of the Framework, taking up Regulation 46, refers to a track record of accreditation and quality audits in at least two consecutive quality audits relating to the types of programs for which the status is requested. The new Policy on Eligibility to apply for Self-Accrediting Status requires two consecutive renewals of accreditation for the same programme, to be met separately for each MQF level included in the scope.

Two quality audits and two renewals of the same program are not automatically equivalent concepts. In a system of multi-year cycles, the difference can result in a much longer vesting period before the institution can access the status. In addition, Regulation 46, as taken from the Standard of Evidence, retains an exceptional route for institutions without a previous track record of provision in Malta when a detailed plan and adequate resources demonstrate a high probability of operating to comparable standards. The new eligibility policy, on the other hand, states that all requirements must be met for the application to proceed. This report also needs a formal explanation and greater transparency.

A transition that should leave no gray areas

The three policies of 21 August are already operational for applications submitted after their publication, while the Outcomes Framework is still in consultation. This temporal overlap creates a concrete issue for institutions that have already started, are preparing or will soon present accreditation, renewal, Level 8, Self-Accrediting or University Status procedures.

A predictable regulatory system should clarify which methodology will apply to procedures already started, whether the new Framework will be able to affect EQAs initiated under previous rules and how providers who are in the intermediate period between the entry into force of the new eligibility policies and the final adoption of the outcome criteria will be treated. Transparency also requires transition rules, not just outcome criteria.

What MFHEA should clarify

The point is not to eliminate the professional judgement of the Panel, but to make it structured, motivated and verifiable. A possible evolution of the Framework would be the introduction of a published judgement rubric, integrated into the manuals of providers and programme accreditation, which first clarifies what it means, in terms of evaluation, that a Minimal or Performance Indicator is mandatory. The rubric should then explain which factors distinguish Full of Substantial, Substantial from Partial and Partial from Non-Compliant, including at least materiality, severity, systemic impact, student impact, compliance history and remendability.

For Partial and Non-Compliant judgments, it would also be advisable to request a specific reasoned judgment, capable of indicating which indicators are not met, what evidence has been used and why those deficiencies justify that rating and not the one immediately higher or lower. A formal moderation control by the QAC on the consistency and sufficiency of the motivation would further strengthen consistency without replacing the professional autonomy of peer reviewers.

Finally, the same logic should be reflected in follow-up and appeal. A provider needs to know what criteria they are trying to meet during remediation and what criteria they can challenge if they believe the rating is not properly supported. Transparency, in this sense, is not an accessory element of the Framework: it is the condition that makes the regulatory consequences that derive from that rating proportionate, controllable and truly defensible.

Next in-depth analysis: how other European systems do it

The issue does not only concern Malta. In other European quality assurance systems, the need to ensure consistency without eliminating the professional judgement of peer reviewers has been addressed through more structured criteria for the formation of judgements, aggregation rules, moderation mechanisms and more explicit obligations of motivation. These models show that the choice is not between professional discretion and numerical automatism: there is a third way, that of structured professional judgement.

In the next in-depth study, Malta Quality Education will compare the model proposed by MFHEA with some European experiences, in particular the Estonian institutional accreditation system and the Dutch NVAO model. The comparison will serve to verify how these systems govern the transition from evidence to rating, the role of materiality and remediation, the motivation of the Panel, moderation and the effective reviewability of decisions, with the aim of identifying which solutions could strengthen the Maltese system without renouncing the necessary autonomy of peer review.

A consultation to be used to the end

The new Outcomes Framework contains positive elements: it makes the relationship between EQA outcome and regulatory consequence more visible, formalizes follow-up and tries to give greater coherence to procedures that ENQA and EQAR had judged to be insufficiently uniform. But its success will depend on its ability to make transparent not only what happens after a rating, but also how that rating is formed.

For providers, the most useful feedback will therefore not be a generic answer about the clarity of the tables. It will be an analysis capable of questioning the entire chain: mandatory nature of indicators, evidence, panel judgement, materiality, level of risk, regulatory decision, follow-up, appeal and impact on future progression. It is in this chain that transparency must become substantial, because a predictable decision is only truly fair when it is also justifiable, verifiable and contestable.