Privacy-preserving human-AI dialogue systems – Organisation of a technological challenge

This opportunity is a call for proposals under the European Defence Fund for advancing privacy-preserving human-AI dialogue systems focused on defence applications. The initiative aims to establish a testing environment and organize a technological challenge to evaluate these systems, specifically for handling classified information and ensuring effective communication through standardized metrics and testing protocols. Eligible applicants must form consortia comprising at least two legal entities from different EU Member States or EDF Associated Countries, ensuring independent operational status without any single parent organization's control over multiple entities. The financial support provided is classified as lump sum grants, covering all eligible costs linked to the proposed activities. The geographical scope includes EU Member States and associated countries, explicitly mentioning Norway. This initiative emphasizes significant collaboration while focusing on sectors such as defence and artificial intelligence. The primary goal of the call is to enhance the trustworthiness and functionality of human-AI dialogue systems through rigorous testing. The budget allocation for this specific grant is set at €7,000,000, with a single-stage application process that opens on 18 February 2025 and closes on 16 October 2025. Proposed activities must relate to creating infrastructure for testing, developing data annotation guidelines, organizing evaluation campaigns, and measuring system performance without sacrificing compliance with military requirements. Among the expected outcomes, there is a push for improved clarity on system performance metrics and valuable data availability to optimize dialogue systems further. The overall objective is to create a robust framework that supports research teams in their efforts, enhancing operational decision-making through reliable human-AI dialogue solutions in the defence domain. The selected projects will be pivotal in shaping a standardized evaluation approach, fostering community engagement across European defence sectors, and optimally utilizing AI technologies for critical defence applications.

This is a call for proposals under the European Defence Fund (EDF) focusing on privacy-preserving human-AI dialogue systems. The call aims to establish a testing environment and organize a technological challenge to evaluate the performance of such systems, particularly for defence use cases involving classified information. The call is structured as a research action implemented via lump sum grants. The main objective is to improve the trustworthiness and progress of Human-AI dialogue systems by subjecting them to standardized tests using shared data, clear metrics, and protocols. This initiative seeks to address the current lack of established metrics for measuring system performance and the potential for errors in these systems. The scope of the proposals should cover the organization of a technological challenge on human-AI dialogue, based on a preliminary evaluation plan detailed in Annex 4 of the call document. Key activities include the collection, annotation, and distribution of data, the development of evaluation plans and metrics, the measurement of system performances, and the organization of debriefing workshops. The challenge is intended to be open to research teams supported by another call topic (EDF-2025-LS-RA-CHALLENGE-DIGIT-HAIDP) and potentially other funding sources. The involvement of representative defence users is crucial for defining use cases, contributing associated data, testing demonstrators, and providing feedback. Eligible activities include: Activities that aim to create, underpin, and improve knowledge, products, and technologies, including disruptive technologies, which can achieve significant effects in the area of defence (generating knowledge) Yes (optional) Activities that aim to increase interoperability and resilience, including secured production and exchange of data, to master critical defence technologies, to strengthen the security of supply or to enable the effective exploitation of results for defence products and technologies (integrating knowledge) Yes (mandatory) Studies, such as feasibility studies to explore the feasibility of new or upgraded products, technologies, processes, services and solutions Yes (optional) Design of a defence product, tangible or intangible component or technology as well as the definition of the technical specifications on which such design has been developed, including partial tests for risk reduction in an industrial or representative environment Yes (optional) System prototyping of a defence product, tangible or intangible component or technology No Testing of a defence product, tangible or intangible component or technology No Qualification of a defence product, tangible or intangible component or technology No Certification of a defence product, tangible or intangible component or technology No Development of technologies or assets increasing efficiency across the life cycle of defence products and technologies No Mandatory activities include: Setting up the infrastructure for testing human-AI dialogue systems. Elaborating data annotation guidelines, collecting and annotating data, assessing quality, and distributing and curating databases. Organizing evaluation campaigns, including coordinating exchanges with participating teams, elaborating evaluation plans, managing experimental test campaigns, objectively measuring system performances, and organizing debriefing workshops. Proposals should describe work packages, tasks, and deliverables that allow for a clear assessment of work package completion. This includes producing detailed evaluation plans agreed upon by stakeholders, annotated databases for evaluations, measurements for all systems tested, and organizing necessary events. Functional requirements include enabling the measurement of dialogue system performances according to detailed evaluation plans. Proposals should describe scenarios, languages covered, the nature and volume of data annotation, a detailed test campaign plan, and evaluation procedures. A user board of representative defence users must be set up for live tests, and their efforts in testing demonstrators and providing feedback should be described. Detailed evaluation plans should be prepared for each evaluation campaign and submitted for discussion with participating teams. The evolution of evaluation plans should consider technical possibilities, cost, scientific relevance, and representativeness of metrics and protocols with respect to military needs. The user board and participating teams should be involved in steering the technological challenge, and proposals should describe governance and decision-making processes. The expected impact includes: Standardization of testing for dialogue systems. Enhanced clarity on the performances of dialogue systems for all stakeholders. Community building at the European defence level. Trustworthy dialogue systems that enhance operational decision-making. Availability of databases to further develop dialogue systems. The call specifies two funding opportunities: EDF-2025-LS with a budget of €7,000,000 EDF-2025-LS-RA-CHALLENGE-DIGIT-HAIDP-STEP with a budget of €20,000,000 Both have a single-stage submission process, opening on 18 February 2025 and closing on 16 October 2025. Conditions for proposal admissibility, eligible countries, financial and operational capacity, evaluation and award processes, and legal and financial setup are detailed in the call document and related annexes. Applicants can find partners through the Funding & Tenders Portal. For help, contact DEFIS-EDF-PROPOSALS@ec.europa.eu. In summary, this call invites proposals to create a robust testing and evaluation framework for human-AI dialogue systems in the defence sector. It emphasizes the importance of standardized metrics, user involvement, and data-driven assessment to enhance the trustworthiness and effectiveness of these systems. The selected projects will organize a technological challenge, providing a platform for research teams to test their systems and contribute to the development of reliable AI-driven dialogue solutions for defence applications.

Summary:

This opportunity is a call for proposals under the European Defence Fund (EDF) for research actions focused on privacy-preserving human-AI dialogue systems. The goal is to set up a testing environment and organize a technological challenge to evaluate the performance of such systems for defence use cases, including their abilities to manage classified information and justify their answers. The call is specifically targeted at research teams and consortia from EU member states and Norway, with a focus on integrating knowledge and improving the interoperability and resilience of defence technologies. The funding is provided as a lump sum grant, with two budget allocations available: €7,000,000 and €20,000,000. The application process is single-stage, with a deadline of October 16, 2025. The project aims to standardize testing for dialogue systems, enhance clarity on system performances, build a community at the European defence level, and make available databases to further develop dialogue systems. Eligible activities include creating knowledge, integrating knowledge, studies, and design, but exclude system prototyping, testing, qualification, certification, and development of technologies increasing efficiency across the life cycle of defence products and technologies. The proposals should address the organization of a technological challenge on human-AI dialogue based on a preliminary evaluation plan, including data collection, annotation, distribution, evaluation plans, metrics, performance measurement, and debriefing workshops.