24
Deferred
AI Safety Institute's limited access to unreleased models undermines safety testing capabilities.
Conclusion
However, we are concerned by suggestions that the Institute has been unable to access as-yet unreleased AI models to perform the pre-deployment safety testing it was set up to undertake. If true, this would undermine the delivery of the Institute’s mission and its ability to increase public trust in the technology.
Government Response Summary
The government stated the AI Safety Institute (AISI) has tested models both before and after deployment and is actively engaged in safety testing. However, it states it would be counterproductive to identify developers with whom access has not yet been secured due to commercial sensitivity and ongoing complex negotiations.
Paragraph Reference
90
Government Response
Deferred
Government Response
Deferred
HM Government
Deferred
Our intention is for legislation to put the AI Safety Institute (AISI) on a statutory footing and there will be further detail about this in the coming months. We are proud of the work of the Institute and what it has achieved in the last year. Putting the Institute on a statutory footing would strengthen its role leading voluntary collaboration with AI developers and leading international coordination of AI safety. This work is vital as the technology continues to develop at pace. Over the last year, AISI has tested models from major labs both before and after deployment. It is actively engaged in safety testing of frontier AI models, working collaboratively with developers who have committed to responsible AI development. Competition is fierce in the frontier AI market, and the nature and timing of releases are highly commercially sensitive. Because of this, and to maintain confidence in our collaborative relationships, it is often not appropriate or possible to provide commentary on individual models under evaluation. This approach helps maintain trust with industry partners and ensures we can fulfil our safety testing mandate. We will continue to share appropriate information about our testing activities through official public announcements, particularly when developers consent to disclosure or when information is already in the public domain. Earlier this year AISI agreed to work in partnership with the US AISI and build towards interoperability between the two Institutes. Building on this. several labs have provided access to their models pre-deployment. The Government can confirm that AISI’s research encompasses critical areas including potential societal harms, misuse risks and autonomy risks. When safety concerns are identified through our testing processes, we work constructively with developers to address these issues. AISI maintains detailed records of all testing outcomes and subsequent modifications, though these details remain confidential to protect commercial interests and our working relationships with AI companies. It is important to remember that it is the AI developers’ responsibility to ensure their models are safe. Our focus remains on ensuring rigorous safety evaluations while respecting commercial confidentiality. AISI continues to make progress in securing access agreements with frontier AI developers for both pre- and post- deployment testing. We have established productive working relationships with multiple developers since last year, and we are expanding these arrangements to enable joint testing capabilities with international partners, particularly through our collaboration with the US AI Safety Institute as outlined in the UK-US Memorandum of Understanding. At this stage, we believe it would be counterproductive to identify specific developers with whom we have not yet secured testing access. We are actively engaged in complex negotiations regarding access to commercially sensitive systems and capabilities. These discussions require careful consideration of intellectual property, technical requirements, and security protocols. While some negotiations take longer than others, we are seeing positive movement toward expanded cooperation with frontier AI labs. Government response to Committee conclusions and recommendations 27 – 31
Source
Report
Third Report - Governance of artificial intelligence (AI)
28 May 2024
HC 38
Addressee Bodies
Department for Science, Innovation and Technology
Timeline
Recommendation age
2.0 yrs
Report published
28 May 2024