14 Bias and source quality
14.1 1. Sources of bias in training data
One of the most problematic aspects of using AI in research processes is the bias inherent in training data.
The data of training of models are never neutral, but reflect quantitative and qualitative imbalances that derive from their very composition.
The prevalence of certain languages, geographical areas or disciplines leads to an over-representation of specific contexts, while other perspectives remain marginal or absent.
👉 The result is partial knowledge, which risks presenting itself as universal despite being based on incomplete foundations.
Alongside these quantitative imbalances, AI systems incorporate implicit cultural biases.
The data reflect the values, conventions and symbolic hierarchies of the communities that generated them.
When these perspectives are assumed to be neutral, algorithms end up replicating gender stereotypes, ethnocentric views or ideologically oriented representations, masking them behind the supposed objectivity of statistical calculation.
A further source of criticism lies in methodological biases.
The collection, selection and normalisation of data respond to criteria that are not always made “explicit”, but which directly influence the outcomes of training.
Choices relating to sampling methods, corpus cleaning or the definition of conceptual categories determine an invisible structure that guides the model’s inferences.
These different levels of distortion converge to generate a significant epistemic risk, i.e. the possibility that scientific research is based on already compromised knowledge, without these limitations being immediately recognisable.
The apparent authority of AI-generated texts can conceal systematic imbalances, imposing on the Academic Community the need to develop critical monitoring tools.
The issue of the traceability of algorithmic decisions therefore takes on importance. Audit logs, fairness metrics and data documentation practices are fundamental tools for making the path leading to a given output recognisable and verifiable.
These mechanisms do not solve the structural problems of datasets, but they introduce elements of accountability that allow researchers to evaluate the consistency of results with criteria of transparency and fairness.
Another critical element concerns the nature of the content produced by generative systems, which may include non-existent bibliographic references, invented citations, or distorted summaries of articles that have actually been published.
These phenomena, often referred to as algorithmic “hallucinations”, directly undermine scientific credibility if they are not promptly recognised and corrected.
The problem of bias, therefore, cannot be reduced to a technical issue, but also has ethical and political implications.
Research risks consolidating inequalities already present in society if shared criteria of inclusivity and fairness are not developed in the selection and management of data.
The GAI, from a potentially emancipatory resource, can become a device of exclusion if left unchecked.
👉 For this reason, alongside individual critical vigilance, it is essential to promote institutional policies and editorial guidelines that guarantee the reliability of sources, the transparency of datasets and respect for the principles of pluralism and neutrality that underpin the scientific community.
14.1.1 Further Readings
See Types of bias in AI models Article on bias in AI - (data/dev/interaction bias)
See Quantification of bias in pre-existing content USC analysis of “common” facts used by AI
See Bibliographic hallucinations Nature research on citations invented by ChatGPT
See Frequency of hallucinations in scientific texts Study on the rate of false citations in psychology
See Open-source tools for managing and measuring bias AI Fairness 360 Toolkit (IBM)
See Generative image production analysis Bias in Generative AI
See AI and SSH Can Generative AI improve social science?
See Bias embedded in training data On the Dangers of Stochastic Parrots
14.2 2. Strategies for the traceability and verifiability of algorithmic decisions
One of the key issues in the use of AI systems in scientific research, is the ability to guarantee the reconstruction and validation of the decision-making processes that lead to a given output.
Unlike human work, where the analysis and inference phases can be explicitly justified and discussed, algorithms operate through chains of calculations that are often opaque and difficult to interpret, even for the developers themselves.
👉 To reduce this lack of transparency, it is necessary to introduce tools that allow the model’s performance to be systematically documented, monitored and evaluated.
In this perspective, audit logs represent a first fundamental mechanism.
They consist of udetailed records of the steps taken by the algorithm during data processing, from the input phase to the generation of the output, including any intermediate transformations.
These records not only allow the path followed by the system to be traced retrospectively, but also identify any critical issues, such as the use of partial sources or the application of undeclared selection criteria.
Alongside process documentation, there is the issue of verifying the fairness of results. Algorithms are not just calculation tools, but devices that incorporate and convey specific hierarchies of relevance.
Fairness metrics have been developed precisely to make these dynamics “measurable”, translating dimensions often considered qualitative, such as inclusion, representativeness or non-discrimination, into numerical parameters.
Through comparative indicators, for example, it is possible to verify whether a model tends to favour certain categories of data or users over others, and whether these imbalances are statistically significant.
👉 The joint application of audit logs and fairness metrics is not limited to technical monitoring, but opens up the possibility of developing shared responsibility in AI management.
Finally, it is important to emphasise that traceability and fairness assessment cannot be understood as occasional or ancillary practices, but must become an integral part of research procedures.
The systematic adoption of these tools is a necessary condition for preserving the reliability of scientific knowledge in the age of automation, preventing the complexity of generative models from translating into a deficit of critical control.
14.2.1 Further Readings
See From Data to Insight: Why Traceability is Crucial for AI Success
See The Rise of AI Audit Trails: Ensuring Traceability in Decision-Making
See What is AI Traceability? Benefits, Tools & Best Practices