Comparisons with Estonia and the Netherlands show that professional judgement can remain central without sacrificing public criteria, motivation, moderation and proportionality.

In the first in-depth study dedicated to the MFHEA consultation on the new Criteria for Outcomes & Follow-up Procedures Framework, we asked a preliminary question with respect to the Low, Moderate, High and Severe thresholds: whether the applicable Minimal Indicators and Performance Indicators are defined in the MFHEA manuals as mandatory levels of achievement and “must be met”, through which methodology the Panel transforms their possible non-satisfaction into an EQA judgment of Fully Compliant,  Substantially Compliant, Partially Compliant or Non-Compliant?

The question is not secondary. The new Framework regulates the consequences of the rating with increasing precision but does not clarify with the same precision the process that produces that rating.

To understand whether this gap is inevitable, or whether it can be addressed without turning peer review into a mechanical exercise, Malta Quality Education examined two European systems operating within the framework of the European Higher Education Area: the institutional accreditation of the Estonian Quality Agency for Education (HAKA) and the accreditation framework of the Nederlands-Vlaamse Accreditatieorganisatie (NVAO) in the Netherlands.

The comparison does not serve to identify a model to be imported into Malta. It serves to verify how other ESG-based systems face the same tension between mandatory criteria, professional judgement and regulatory consequences.

The result is significant. In both benchmarks, the technical discretion of peer reviewers remains essential, but it is made more structured through rules governing the transition between evidence, judgement, outcome and follow-up. The point, therefore, is not to choose between the autonomy of the Panel and numerical predetermination. The point is to build an EQA judgment that is professional, but also sufficiently predictable, motivated and reviewable.

The MFHEA node: mandatory indicators and four levels of compliance

The MFHEA Provider Accreditation Manual states that Minimal Indicators reflect a “mandatory level of achievement” and must be met at licensing and during each audit; Performance Indicators are also mandatory and must be met from the first cyclical review onwards. The same approach is used in the Programme Accreditation Standards and Procedures. The consultation document also confirms that, for renewal, University Status, Self-Accrediting Status and Extension of Scope procedures, the assessment must consider both Minimal and Performance Indicators.

This approach produces a methodological tension that the new Outcomes Framework does not directly address. If all applicable indicators are mandatory, what does substantial compliance mean when one or more indicators are not fully met?

If a mandatory indicator that has not been complied with can be absorbed into the overall judgement of the Standard, in what sense should its mandatory character be interpreted?

And, above all, what factors allow the Panel to establish that the deficiency corresponds to Substantial, Partial or Non-Compliance?

The issue is made even more important by the fact that the Framework attributes very different consequences to the four ratings. In provider renewal, a combination of Partial Compliances can lead to Moderate or High Risk; Non-Compliance leads to High Risk. For University and Self-Accrediting Status, a single Partial or Non-Compliance can directly affect the outcome. Consequently, the EQA judgment on the individual Standard is not a descriptive classification: it becomes the input data of a regulatory decision.

Estonia: mandatory criteria, but an explicit methodology to graduate judgement

The HAKA model is particularly interesting precisely because it starts from an assumption similar to the Maltese one: quality criteria are mandatory. In the institutional accreditation procedure approved in April 2025, HAKA clearly distinguishes between quality criteria, which must be respected, and guidelines, which are non-binding and offer flexibility on how to achieve the objectives. The presence of mandatory requirements, however, does not prevent the Estonian system from using a graded judgment.

HAKA adopts a four-level scale at the quality criterion level.

• “Fulfilled” means that all requirements included in the policy are met.
• “Substantially fulfilled” occurs when there are some shortcomings, but the institution has demonstrated, through actions and documents, that it has identified them and is concretely capable of eliminating them.
• “Partially fulfilled” instead presupposes significant shortcomings: the institution has not demonstrated the same consolidated capacity, but the Panel believes that there is a credible potential for remediation, which can also be inferred from the behavior of the institution with respect to other criteria.
• “Not fulfilled” is reserved for cases where fulfilment is deficient and lacks a demonstrated ability to deal with shortcomings.

This methodological choice resolves part of the contradiction that remains implicit in Malta. A requirement can be mandatory, and, at the same time, its degree of fulfilment can be assessed in a graduated way if the system explains what a deficiency means, how significant it is and what role the institution’s ability to correct it plays. In other words, the requirement is not automatically transformed into a binary “met/not met” logic: it is accompanied by public criteria that explain how shortcomings affect the EQA judgment.

Not an algorithm, but structured professional judgement

The most useful fact of the Estonian model is that professional judgement does not disappear. HAKA does not assign a numerical score to each finding and does not produce the result through an automatic formula. However, it makes it “structured”.

After attributing the four ratings to the quality criteria, the procedure establishes aggregation rules to determine the judgement of the broader assessment areas. In some combinations, the conclusion is predetermined; in others, the Panel must analyse strengths and areas for improvement and choose between two possible outcomes.

It is an important distinction. Decision rules delimit the space of discretion without cancelling it. The Panel knows when it can exercise an additional professional assessment and knows what elements it needs to consider. Predictability therefore derives not only from the counting of ratings, but from the combination of public rules and reasoned judgement. This is the point that the Maltese debate should consider: an EQA judgment can be simultaneously professional, evidence-based and bounded by published criteria.

The HAKA model also contains an additional accountability mechanism. Judgements should be reached by consensus; if consensus is not possible, a majority vote is held and dissenting opinions, with their justifications, must be reported in the report. ^[4] This detail is also relevant, because it makes it clear that an assessment can be professionally complex without becoming opaque.

Moderation: who checks that the judgement is sufficiently motivated?

The Estonian system introduces a guarantee that appears particularly relevant to the MFHEA discussion. The final decision on institutional accreditation is taken by the Quality Assessment Council and must be reasoned. But the Council does not limit itself to receiving the Panel’s report: if it identifies contradictions or insufficient justification in the judgements, it can return the report to the Panel for review, clarification and, if necessary, modification.

This mechanism does not authorize the Council to arbitrarily substitute its own judgment for that of the experts. Rather, it introduces a moderation function: before the judgement produces a regulatory decision, someone verifies that the connection between evidence, analysis and rating is sufficiently consistent and motivated. In a system where a single rating can affect the duration of accreditation or access to institutional status, this function becomes an important part of the quality of the process itself.

Applied to the Maltese context, the principle could be simple: when a Panel assigns Partial or Non-Compliant, the report should make explicit which indicators are not met, what the materiality of the deficiency is, why the core purpose of the Standard is compromised to that extent and why that category was chosen instead of the next higher or lower one. The QAC could therefore verify not academic merit in place of the Panel, but the sufficiency and consistency of motivation before the rating feeds the risk matrix.

The provider is not a passive observer of the draft

Another difference concerns the phase prior to the decision. HAKA sends the draft assessment report to the institution and allows comments to be submitted; the Panel must review them and prepare a response letter explaining which observations were considered and which were not, providing explanations and justifications. The Council’s final decision also considers the institution’s timely submitted comments.

The point is not to turn the process into an unlimited adversarial process or to allow the provider to produce new evidence after the visit has concluded. It is a matter of recognizing that, when the transition from findings to judgement can have significant regulatory consequences, the institution should be able to report not only material errors, but also manifest inconsistencies between evidence, criteria and proposed ratings.

This possibility strengthens the reviewability of the EQA judgment and can reduce subsequent litigation. A provider who understands the logic of the rating can accept it, correct any misunderstandings or, if necessary, challenge it on precise grounds. Transparency thus becomes a tool for process quality, not just an obligation to publish.

Follow-up and remediation: severity also counts in Estonia

The comparison becomes even more interesting in the follow-up. When HAKA imposes a secondary condition, the decision must identify the concrete deficiencies and the deadline within which they must be eliminated. In the subsequent verification, the experts distinguish between deficiencies fully, substantially, partially or not eliminated. If a deficiency is only partially eliminated, the Council must consider its severity and can decide whether the condition should be considered fulfilled or not.

This approach prevents follow-up from automatically reducing to a new binary count. Remediation is evaluated for what it has actually produced and in relation to the residual severity of the problem. Here, too, professional judgement is maintained, but it is exercised within an explicit framework.

For MFHEA, the comparison is relevant because the proposed Framework tends instead to link the failure to achieve Full or Substantial in follow-up to very clear consequences. The question is not to ask the Authority to waive severe measures when the deficiency remains material. It is asking that the residual severity, the impact on students and the concrete remendability be a visible part of the decision.

Netherlands: a “weighted and substantiated” conclusion

The NVAO model adopts a different structure from the HAKA and does not have to be mechanically superimposed on the Maltese system. However, it is useful for another reason: the Dutch procedure insists that the conclusion must be “weighted and substantiated”. In institutional audit, each Standard can be evaluated as Meets, Partially Meets or Does Not Meet. The conclusion is then formulated according to public decision rules, but it must still be considered and motivated.

Partially Meets is not treated only as a numerical shortage. NVAO links it to the fact that the Standard is met “to a significant extent”, while requiring improvements. Above all, a conditionally positive conclusion is only possible if the Panel considers it realistic that the necessary improvements can be implemented within two years. If remediation is not realistically achievable, the outcome must be negative. The conditions must also be formulated in concrete terms.

The comparative value of this approach is clear: remendability becomes part of the decision. It is not enough to know how many Standards Partial are; It is necessary to ask whether the problem can be concretely corrected and in what time horizon. This element is particularly useful for the MFHEA debate, because it allows us to distinguish a serious but realistically remediable deficiency from a failure that makes it unlikely that compliance will be restored.

Recommendations and compliance are not the same thing

NVAO also offers an additional cue. In its institutional audit framework, it establishes that recommendations for improvement are indicated separately from the substantiation of judgements. The principle is important because it avoids automatically confusing enhancement and non-compliance.

HAKA uses a similar distinction at the decision-making level: areas for improvement and recommendations that indicate non-compliance and have affected the decision are separated from opportunities for further development, which do not indicate non-compliance and do not influence the outcome.  This distinction makes the regulatory value of each observation more readable.

The comparison is relevant to the new MFHEA framework, which states that unmet recommendations will be considered severe violations. If the recommendations may be different in nature, a general escalation formula risks giving the same weight to a recommendation linked to a material deficiency and to an indication oriented mainly towards enhancement. A more transparent methodology should clearly distinguish recommendations that impact compliance from those that represent opportunities for improvement.

ESG does not prescribe Estonia or the Netherlands, but prescribes the outcome

Neither the HAKA nor the NVAO is a mandatory model for Malta. The Standards and Guidelines for Quality Assurance in the European Higher Education Area do not prescribe a specific scale of ratings, nor do they impose a specific aggregation formula. ESG 2.5, however, requires that outcomes and judgements be based on explicit and published criteria, applied consistently; the Guidelines specify that these criteria must be pre-defined, consistently interpreted and evidence based. ESG 2.6 requires reports capable of linking evidence, analysis and findings, conclusions and recommendations; ESG 2.7 requires clearly defined complaints and appeals.

It is in the light of these principles that the European comparison becomes useful. HAKA and NVAO show two different ways of making EQA judgment more reconstructable without eliminating peer review. The first structures in detail the graduation of criteria, the rules of aggregation and moderation; the second insists on the weighted and substantiated conclusion and on the concrete feasibility of remediation. Both show that consistency does not necessarily have to be achieved through downstream numerical automation.

This observation is particularly pertinent for MFHEA, because ENQA had already recommended to clearly establish, for each accreditation procedure, the criteria followed to reach the final decision and to publish and communicate all EQA criteria of outcomes. EQAR subsequently found that there were no sufficient guiding criteria and a systemic approach to the consistency of decision making. The new Framework addresses part of this need, but the comparison suggests that the work should also include the methodology by which the judgement of the Standard is born.

The real difference: downstream certainty and upstream structure

The result of the comparison can be summarized in a distinction. The proposed MFHEA Framework is strongly downstream structured: once the rating has been assigned to the Standard, the combination of ratings determines the risk level and guides decisions and follow-up. HAKA and NVAO, in different ways, also devote more attention to the upstream part: how the judgement is formed, what elements qualify it, how it is motivated and how the consistency of the transition between evidence and outcomes is checked.

The risk of a very precise downstream but less determined upstream system is a paradox of consistency. Two Panels could read similar deficiencies differently and classify a Standard differently; After that moment, the risk matrix would apply very different consequences in a perfectly uniform way. The uniformity of the consequence, therefore, does not in itself guarantee the uniformity of the judgment that generated it.

For this reason, the heart of the discussion should not only be how many Partial Compliances produce High Risk. First of all, it is necessary to make sufficiently transparent what produces a Partial Compliance. This is where EQA judgment must become structured, reasoned and reviewable.

What MFHEA could do

The comparison does not suggest abandoning the new Outcomes Framework. He suggests completing it. The most coherent solution would be to combine the risk tables with a published Judgement and Decision-Making Rubric, integrated into the Provider Accreditation Manual and the Programme Accreditation Standards and Procedures. Such a rubric should first of all clarify the operational meaning of the “mandatory” nature of the Minimal and Performance Indicators and explain what effects their non-satisfaction produces.

The methodology should then make visible the factors that distinguish the contiguous categories: number of unmet indicators, materiality, severity, systemic or isolated nature of the deficiency, impact on students and academic standards, possible presence of critical indicators, compliance history and credible ability to remediate. The Panel would retain its professional judgement, but would be required to explain why the evidence leads precisely to Substantial, Partial or Non-Compliant.

For each Partial and Non-Compliant it would be advisable to request a reasoned judgement statement: indicators not met, evidence used, nature and materiality of the deficiency, impact, remediability and reason why that category was chosen over the one immediately above or below. This would make the judgement understandable for the provider, verifiable by the QAC and concretely contestable in appeal.

A second element could be a formal moderation function. If the QAC identifies contradictions or insufficient justification in the rating, it should be able to send the report back to the Panel for clarification before applying the risk matrix. The principle does not require QAC to replace the Panel: it requires that the regulatory decision is not based on insufficiently explained judgement.

Finally, MFHEA could more clearly distinguish between compliance recommendations and enhancement recommendations, expand in a controlled manner the provider’s ability to report manifest inconsistencies between findings and proposed judgement before the final decision, and incorporate severity and realistic remediability into follow-up. Numerical thresholds could continue to play a predictability function, but as part of a broader methodology and not as the only indicator of risk quality.

Not less rigour, but a more demonstrable rigour

The European comparison leads to a conclusion that deserves to be distinguished from any request for EQA relaxation. A credible quality assurance system must be able to attribute Non-Compliance when there is a material failure, impose conditions when the risk requires it, and take stricter measures when students or academic integrity are not adequately protected. The European benchmark does not suggest otherwise.

However, it suggests that rigor is more robust when judgement is built through public criteria, when remediation is realistically assessed, when recommendations are not confused with non-compliance, and when the provider can reconstruct the path that led to the decision. In this sense, structured professional judgement does not reduce the authority of the Panel: it makes it more defensible.

For Malta, this could represent the next step in the reform. MFHEA has already started to make outcomes more predictable. The challenge now is to ensure that the EQA judgment that fuels those outcomes is equally transparent, consistent, and substantiated.

Consultation as an opportunity for regulatory design

The ongoing consultation, expiring on 11 September 2026, therefore offers providers a wider possibility than simply commenting on the thresholds. Institutions can help define what kind of decision-making architecture they believe is necessary for the Framework to achieve the stated objectives of transparency, consistency, fairness and proportionality.

The question to ask MFHEA is not whether it should follow Estonia or the Netherlands. It is whether the Maltese system should make explicit, with greater precision, the transition between indicators, evidence and judgement, and what guarantees should accompany that passage when the rating produces important regulatory effects. The European comparison shows that concrete tools exist and that they can coexist with the technical discretion of peer-review experts.

Malta Quality Education believes that this is one of the issues on which the contribution of providers can be particularly useful: not to make quality assurance less demanding, but to make sure that every EQA judgment is understandable, proportionate, justifiable and truly reviewable.

A consultation to be used to the end

The new Outcomes Framework contains positive elements: it makes the relationship between EQA outcome and regulatory consequence more visible, formalizes follow-up and tries to give greater coherence to procedures that ENQA and EQAR had judged to be insufficiently uniform. But its success will depend on its ability to make transparent not only what happens after a rating, but also how that rating is formed.

For providers, the most useful feedback will therefore not be a generic answer about the clarity of the tables. It will be an analysis capable of questioning the entire chain: mandatory nature of indicators, evidence, panel judgement, materiality, level of risk, regulatory decision, follow-up, appeal and impact on future progression. It is in this chain that transparency must become substantial, because a predictable decision is only truly fair when it is also justifiable, verifiable and contestable.