Pacific Grove, February 1975
In July 1974, a group of scientists led by Paul Berg published a letter in Science that no regulator had asked for. Recombinant DNA technology, barely a year old, let researchers splice genetic material from one organism into another, and the letter's signers, several of them among the technique's own inventors, asked colleagues worldwide to voluntarily pause a narrow slice of that work (experiments joining genes for toxin production, or genes from cancer-causing viruses, into fast-growing bacteria) until the field had worked out how much containment was actually needed. Nobody had proven a specific experiment dangerous yet. The request was pause first, study the risk, then resume under agreed rules, a sequence with almost no precedent in a scientific culture that treated open, unrestricted publication as close to a first commandment.
The follow-through came seven months later, at the Asilomar Conference Grounds in Pacific Grove, California. Organized by Berg together with Maxine Singer and Richard Roblin, the four-day meeting in February 1975 brought together roughly 150 scientists from around the world, plus lawyers and journalists invited to keep the discussion honest and public. The group's task was narrow: whether the voluntary moratorium could be lifted, and if so, under what conditions. The participants concluded, not unanimously, that research should proceed, under a graded system that matched physical and biological containment to how much risk a given experiment class actually carried, from ordinary precautions for low-risk work up to the strictest containment for the categories judged most dangerous. Their recommendations went to a National Institutes of Health committee, which turned them into the formal Guidelines for Research Involving Recombinant DNA Molecules, issued in July 1976. The moratorium lifted once that machinery existed to replace it. Recombinant DNA research did not stop in 1975; it resumed inside a structure the scientists who understood the technology best had designed for it, before any legislature had to.
Forty-two years later, in January 2017, more than a hundred AI researchers and specialists in law, economics, and ethics booked the same conference grounds for a meeting they explicitly named after it: the Asilomar Conference on Beneficial AI, organized by the Future of Life Institute, which produced the 23 Asilomar AI Principles. The venue choice was a claim about which historical case the organizers thought their own field's problem most resembled: a technology whose risks were understood earliest and best by the people building it, arriving faster than legislatures could write rules for it, in a field where open publication had been the working default right up until that default needed re-examining. This chapter is about what that resemblance gets right, where it breaks down once the technology is a general-purpose AI model rather than a laboratory technique, and how treaties, national regulators, and AI labs have actually built oversight around the intersection of the two.
Why Asilomar's Design Worked
Three design choices, more than the meeting itself, are why Asilomar is still taught as the reference case for scientist-led self-governance rather than a historical footnote. First, the people writing the rules understood the underlying technology best, and the field trusted that judgment enough to comply with a voluntary pause that no law compelled; a moratorium proposed by an outside regulator with a shallower grasp of the biology would have faced far more resistance and far more argument over whether the risk was even real. Second, Asilomar did not ban recombinant DNA research. It built a risk-tiered system: minimal oversight for low-risk experiment classes, progressively stricter containment for higher-risk classes, and an explicit "not yet" for the narrow set judged too uncertain to proceed on at all, so oversight cost scaled with actual risk instead of falling equally on every experiment in the field. Third, the conference did not end with a communique. It ended with a recommendation that produced a standing federal review process and a written, revisable set of guidelines, so the judgment calls made in four days became a durable institutional practice that could be updated as evidence accumulated, rather than a one-time pledge that would age badly as the science matured.
All three properties, technical credibility of who sets the rules, risk grading instead of blanket prohibition, and a durable institutional process instead of a one-time statement, reappear largely unchanged in how AI labs write their own biosecurity policies today. That is not a coincidence later observers noticed. It is a template the AI safety field borrowed on purpose, down to the choice of conference venue.
The Governance Stack Above Any Single Lab
Asilomar's recommendations shaped how one country regulates federally relevant research. Above and around institutional bodies like it sits a taller stack of governance, and AI-biosecurity policy only makes sense once that stack is visible layer by layer.
At the top sits international treaty law. The Biological Weapons Convention, opened for signature on 10 April 1972 and in force since 26 March 1975 (the same year as Asilomar, by coincidence), was the first multilateral treaty to ban an entire category of weapons outright: it prohibits developing, producing, stockpiling, or acquiring biological agents or toxins "of types and in quantities that have no justification for prophylactic, protective or other peaceful purposes," along with the weapons designed to deliver them. Nearly every country in the world has joined it. But the BWC has a structural weakness a student who only reads its opening articles tends to miss: it has no verification regime. Unlike the Chemical Weapons Convention, which entered into force in 1997 and created the Organisation for the Prohibition of Chemical Weapons with standing authority to conduct "any time, anywhere" challenge inspections, the BWC has never had an inspectorate. States parties negotiated a verification protocol for six years, from 1995 to 2001, through a body called the Ad Hoc Group; in July 2001, at the group's final scheduled session, the United States rejected the draft protocol outright, judging that its inspection provisions would expose commercial and biodefense secrets without reliably catching determined violators. No replacement negotiation has succeeded since. What the BWC has instead is a small Implementation Support Unit in Geneva with an administrative, not investigative, mandate, and voluntary annual Confidence-Building Measure declarations that states file about their own biodefense-relevant activity, unaudited by anyone outside their own government. A treaty banning an entire weapons category, with no one authorized to check whether any signatory is actually complying, is the single most important fact to carry forward about the BWC, and it is why every governance layer beneath it has to do real work rather than defer upward.
The next layer down is national law, in two different forms depending on what it targets. One form controls who may possess or handle a specific dangerous biological agent at all. The United States runs this through the Federal Select Agent Program, jointly administered by the CDC and the US Department of Agriculture under authority Congress created through the Public Health Security and Bioterrorism Preparedness and Response Act of 2002: any institution wanting to possess, use, or transfer an agent on the Select Agents and Toxins list must register, pass a facility security and biosafety review, and have its relevant personnel individually vetted through a federal security risk assessment before anyone touches the material. This is an access-control regime rather than a review of research content: it does not ask whether a specific experiment is wise, only whether the people and facility handling the agent have cleared a security bar.
The other form of national law reviews proposed research itself, tier by tier, much closer to what Asilomar's recommendations built in the United States. India runs this through a three-tier structure that traces its institutional logic directly back to that same template. Genetically engineered organisms are regulated under the Environment (Protection) Act, 1986, through rules dating to 1989: the Institutional Biosafety Committee (IBSC) at the individual research institution is the first checkpoint any recombinant DNA proposal has to clear; the Review Committee on Genetic Manipulation (RCGM), housed in the Department of Biotechnology, reviews higher-risk categories and confined field trials nationally; and, for the highest-stakes decisions, particularly environmental release, the Genetic Engineering Appraisal Committee (GEAC), under the Ministry of Environment, Forest and Climate Change, is the apex authority. An institution-level committee for routine review, a national technical body for harder cases, an apex authority for the highest-consequence decisions: this is Asilomar's tiered, insider-credible, standing-committee logic, reproduced almost exactly, on a different continent, decades later. It is worth being precise about what this apparatus does not cover: nothing in the IBSC-RCGM-GEAC chain was written with the possibility in mind that biological design assistance might come from a general-purpose AI system a researcher is chatting with rather than a colleague down the hall, and as of this writing India has not publicly layered an AI-specific rule onto that apparatus the way some governance responses described later in this chapter attempt to.
A third layer sits between national research-oversight law and the treaty layer above it: export and access controls on tools and materials themselves, rather than on research conduct. The Australia Group, an informal arrangement of roughly 40 countries formed in 1985, is not a treaty and creates no binding legal obligation; it works by getting member governments to voluntarily align their own national export-control lists for chemical and biological agents, equipment, and related technology. A more targeted version of the same logic operates inside the DNA-synthesis industry: companies that manufacture custom DNA to order have, since 2009, coordinated through the International Gene Synthesis Consortium to screen incoming orders against databases of sequences of concern before fulfilling them. That screening approach assumes a dangerous order can be caught by matching the requested sequence against a list of already-known dangerous ones, an assumption AI-based biological design tools put direct pressure on, since a tool proposing genuinely novel sequences will not necessarily resemble anything already on a watch list. In October 2023, the US executive order on AI directed the White House Office of Science and Technology Policy to build a strengthened nucleic-acid-synthesis screening framework in response; OSTP published it in 2024, explicitly citing AI-enabled biological design tools as a reason the older, sequence-matching-only approach needed reinforcing. That specific gap, and how AI labs themselves have started responding to it from the other end, upstream of any synthesis order, is where the rest of this chapter goes.
How AI Labs Wrote Asilomar's Logic Into Their Release Policies
By late 2023, the leading frontier AI labs, Anthropic, OpenAI, and Google DeepMind foremost among them, had each published a policy doing, for their own models, something structurally close to what Asilomar's recommendations built for recombinant DNA proposals: a standing internal review process that grades a model's capabilities into risk tiers before release and attaches specific, named safeguards to each tier, instead of either releasing everything by default or refusing to release anything.
Anthropic's Responsible Scaling Policy, first published in September 2023, is the most explicit about the borrowing: it defines a ladder of AI Safety Levels (ASL) the policy itself describes as modeled loosely on the biosafety-lab BSL containment levels that trace back to Asilomar's own recommendations. ASL-2 covers the general capability range of ordinary publicly available frontier models; ASL-3 requires enhanced deployment and security safeguards, and the policy commits Anthropic to have those specific safeguards implemented before a model may be released at that tier, or to hold back deployment of the relevant capability until they exist. Biological, chemical, radiological, and nuclear (CBRN) weapons uplift, of which biological risk is the most developed sub-category in current evaluation practice, is one of a small number of catastrophic-risk domains the policy tracks explicitly, alongside cyber capability and model autonomy. OpenAI's Preparedness Framework tracks a similar named set, cybersecurity, CBRN, persuasion, and model autonomy, and states plainly that only a model scoring "medium" risk or below on each category may be deployed at all, reserving further capability development itself for models scoring "high" or below. Google DeepMind's Frontier Safety Framework defines its own CBRN category as the risk of a model assisting in the development, preparation, or execution of a chemical, biological, radiological, or nuclear threat, gated by comparable thresholds it calls Critical Capability Levels. All three documents follow the same idea, developed independently: name the catastrophic-risk categories in advance, evaluate every major model against each one before release, and pre-commit to what happens if a result crosses the line.
Two features of this design are exactly Asilomar's choices, transplanted rather than reinvented. These are self-governance commitments each company wrote for itself; no external regulator currently requires them and no external body audits compliance, only reputation, customer and investor scrutiny, and regulators who increasingly cite these frameworks as evidence of what due diligence should look like when drafting the next round of binding law. And like Asilomar, none of them bans a whole category of AI capability outright. They grade: a specific evaluation result crossing a specific threshold triggers a specific, pre-committed response, the same risk-tiered logic India's IBSC-RCGM-GEAC chain has run for genetically engineered organisms since 1989.
The specific safeguard these policies reach for most often, once a capability crosses a concerning threshold, is not "do not deploy the model." It is structured access: releasing a capability through a controlled channel, typically an API the lab itself operates, that can enforce usage restrictions, monitor for misuse, and rate-limit or revoke a specific customer's access, rather than an unrestricted public release. The term comes from AI governance researcher Toby Shevlane's 2022 paper, "Structured Access: An Emerging Paradigm for Safe AI Deployment," and the distinction it draws matters more here than almost anywhere else in AI policy: structured access is a revocable decision. If a lab discovers six months after release that a capability is being misused, it can tighten a restriction or cut a customer off, because the model never left its own infrastructure. Open release of a model's weights is not revocable in that sense: once published, every already-downloaded copy keeps running under whatever restrictions its holder chooses to honor, and no lab can reach into someone else's computer to add a safeguard afterward. This is precisely why a capability tier judged safe under structured access does not automatically clear the far higher bar for open-weight release: the two decisions are not answering the same question, because only one of them can be undone.
The Problem a Single Publication Decision Can No Longer Solve
Asilomar's model, and the review chains built on it, share one assumption that is easy to miss until it is pointed out directly: the thing being governed is a discrete, countable unit. A specific experiment, reviewed once, before it happens. A specific paper, which either gets published with a sensitive detail included or does not. Governance built for that assumption asks a clean question with a clean answer: should this one thing proceed, yes or no, and who decides.
A general-purpose AI model breaks that assumption in a specific, structural way. The relevant capability does not live in one experiment or one document; it lives inside a system queryable in an effectively unbounded number of ways, by an effectively unbounded number of people, most of whom the lab that built it will never identify or vet. A paper's sensitive content, once identified, can be redacted, and the redaction is the whole fix. A model's capability cannot be "redacted" the same way, because nobody, including the lab that trained it, can fully enumerate every sequence of ordinary-looking prompts that might reach it before someone actually finds one. And publishing a paper is a single event with a single moment where it can be stopped; deploying a model at API scale, to millions of users, is closer to opening a door than to publishing a document, because restricting it again means acting against millions of individually invisible interactions rather than simply not printing one document.
This is exactly why structured access, capability evaluations run before deployment rather than judged after the fact by a single editor, and tiered mitigations attached to specific thresholds exist as machinery Asilomar's own framework never needed. Asilomar's governance question was: should this experiment happen. The AI-era question layered on top is: given that a capability now lives inside a system many people can reach, what access controls, monitoring, and reversibility guarantees have to be in place before any of them can reach it. That is a harder problem, not simply a bigger version of the same one, and it is why powerful bio-design tools specifically are increasingly gated by know-your-customer verification, tying access to a vetted, identifiable account rather than an anonymous key, extending the Select Agent Program's own logic for physical laboratory access up into the software layer above it.
Common Misconception
Students who have just studied Asilomar tend to reason as follows: dangerous biological knowledge has always been managed by not publishing it, so the AI-era version of that policy must simply be not training the risky capability into a model, or not releasing the model that has it, and the problem is solved the way it always has been. This is wrong, for exactly the reason the previous section laid out.
"Do not publish the sensitive detail" works for a paper because a paper is a single artifact with a single publish-or-not decision, and once made correctly, the risk is fully addressed; there is no second place the same information could leak from. A general-purpose AI model does not have that property, even in principle. The same underlying capability that shows up on a targeted biological-risk evaluation can, in a large enough model, be reachable through combinations of ordinary, individually harmless-looking prompts no single publication decision ever touches, can vary in how easily it surfaces depending on exactly how a request is phrased, and can behave differently across fine-tuned copies, jailbroken variants, or later versions built on the same base model. There is no single "the paper" to decide not to publish, only a queryable system, reachable by many people through many paths, whose behavior has to be tested, bounded, and continuously monitored rather than settled once by a single editorial yes-or-no. That is the whole reason the RSP-style frameworks above do not stop at "evaluate once and decide whether to publish": none of their added machinery, structured access, repeated evaluation, know-your-customer gating, would be necessary if "do not publish the risky part" were still sufficient on its own.
Worked Example: A Capability Evaluation Crosses the Line
Consider a hypothetical, built directly from the structure that Anthropic's RSP, OpenAI's Preparedness Framework, and Google DeepMind's Frontier Safety Framework each publicly describe. No named lab's model has publicly been confirmed to cross this specific threshold as of this writing; the value of walking through it is that the governance steps it triggers are not hypothetical at all. They are the actual committed process written into these public policies today.
An AI lab, call it Aurora Labs, has a new frontier model, internally code-named Theta, scheduled for public release in six weeks. Aurora's responsible scaling policy requires a battery of capability evaluations before any major release, one of which specifically tests whether the model provides meaningful uplift on tasks relevant to biological weapons. What that evaluation actually measures, and how, is set by biosecurity specialists working under strict information-security controls; neither the evaluation's content nor its detailed results are things this chapter, or Aurora's own public documentation, would ever publish. Theta's score comes back above the threshold Aurora's policy has pre-defined for its planned deployment tier.
The first consequence is procedural and automatic: Theta's release date is suspended, without any human first having to judge how urgent the launch feels. Second, the result is escalated to whatever accountable safety authority Aurora's policy names in advance, a role Anthropic's own RSP calls a Responsible Scaling Officer, and which other labs structure as a cross-functional safety review body, so nobody has to decide under launch-week pressure who gets to make the call. Third, because a single evaluation run can be noisy, an artifact of how one prompt happened to be phrased, the policy requires independent confirmation: a red-team or a second evaluation team, working separately from whoever ran the first test, tries to reproduce and further probe the result under harder conditions before Aurora treats it as real.
Suppose confirmation comes back positive. The decision now forks, and this is the fork that does the actual governance work. If the confirmed capability sits within a tier for which Aurora's policy has already defined safeguards, tighter usage monitoring, stricter rate limits, verification for accounts requesting the sensitive capability, structured API-only access, Aurora must implement them before Theta can ship; release is delayed, not cancelled, resuming once the safeguards are verified as actually in place. If instead the result exceeds what any currently defined tier covers, the policy commits Aurora to something stronger: pause further scaling or deployment of that capability until new, adequate safeguards are designed, which typically takes longer, because nobody had needed to specify in advance what "adequate" means at a level nobody expected to reach yet. Either way, the evaluation result, confirmation process, and safeguard decision get documented, typically in the model's public system card, since Aurora's own future evaluations need a baseline to compare against, and regulators and outside researchers increasingly treat these disclosures as the evidence for whether a lab's self-governance is real or merely stated.
What Theta's story does not include, anywhere, is a single publication decision. The entire process is about access, confirmation, and reversibility, exactly the machinery the previous two sections argued a general-purpose model requires that a single research paper never did. The diagram below traces this same decision path.
Active Recall
Attempt each question before reading its answer.
- What three design features explain why Asilomar succeeded as voluntary self-governance, and specifically why does a blanket ban on a dual-use technology typically fail to achieve the same thing?
- A researcher in Bengaluru wants to conduct recombinant DNA research. Trace the institutional review pathway her proposal must pass through under India's biosafety framework, from first review to the international treaty layer that sits above the entire national system.
- Explain the structural difference between how the Biological Weapons Convention is enforced and how the US Federal Select Agent Program is enforced. Why does this difference matter for verification?
- True or false, with justification: "Since a general-purpose AI model is not itself a published paper, none of the lessons from Asilomar-style self-governance transfer to it."
- Walk through, step by step, what happens under a responsible-scaling-style framework when a pre-deployment biological-risk evaluation for a new model returns a result above the policy's defined threshold, up to the point where the model is or is not released.
- Structured access (API-only) and open-weight release are often described as points on the same spectrum rather than a binary choice. Why does a policy that permits structured release of a capability at a given tier not automatically justify open-weight release of that same capability at that same tier?
Answers.
1. Insider credibility (rules written by people who understood the technology, so the field trusted and complied with a voluntary pause no law compelled), risk-tiering rather than blanket prohibition (oversight cost matched to actual risk, so low-risk work could continue immediately instead of every experiment being held to the standard of the most dangerous one), and conversion into a standing, revisable institutional process rather than a one-time pledge, so the framework could be updated as evidence accumulated. A blanket ban typically fails at the second point: it treats every use of a dual-use technology as equally dangerous, wasting oversight capacity on genuinely low-risk work while giving researchers a strong incentive not to comply, since compliance costs the same regardless of how careful any specific proposal actually is.
2. Her proposal is first reviewed by her own institution's Institutional Biosafety Committee (IBSC), under the rules issued in 1989 pursuant to the Environment (Protection) Act, 1986. Depending on the risk category, it may then need review by the Review Committee on Genetic Manipulation (RCGM), housed in the Department of Biotechnology, and, for the highest-stakes decisions such as environmental release, final appraisal by the Genetic Engineering Appraisal Committee (GEAC), under the Ministry of Environment, Forest and Climate Change. This entire national chain sits underneath the Biological Weapons Convention, which prohibits the development of biological weapons outright but does not itself review or approve any individual experiment; it sets the outer legal boundary within which India's domestic framework operates.
3. The BWC is an international treaty with no verification regime: no inspectorate, no legal authority to conduct on-site checks, and no external body auditing whether any signatory is actually complying, only a small Geneva-based Implementation Support Unit with an administrative mandate and voluntary, self-reported, unaudited annual declarations. The Federal Select Agent Program, by contrast, is a domestic access-control regime with direct enforcement teeth: any US institution seeking to possess a listed agent must register with the CDC or USDA, pass a facility and biosafety review, and have specific personnel individually vetted through a federal security risk assessment before ever touching the material, all of which a national regulator can actually check and act on. The difference matters because verification strength depends on which layer of the governance stack is doing the checking: the treaty layer is comparatively weak precisely because no state has agreed to let outside inspectors verify its compliance, while the national layer beneath it, where a government regulates its own institutions, can be considerably stronger.
4. False. What transfers is not the specific artifact being governed (a paper versus a model) but the underlying design logic: rules written by people with real technical credibility in the risk, oversight graded to match actual risk tier rather than applied as a blanket restriction, and a standing, revisable institutional process rather than a one-time decision. Anthropic's RSP, OpenAI's Preparedness Framework, and DeepMind's Frontier Safety Framework all reproduce exactly this logic applied to a different kind of artifact. What does not transfer, because the artifact genuinely changed, is the mechanism of "do not publish it": a paper's risk can be fully addressed by a single publish-or-not decision, while a model's capability is reachable through many unpredictable paths by many unvetted users and cannot be secured by any single decision of that kind, which is why the AI-era frameworks add structured access, repeated evaluation, and post-deployment monitoring that Asilomar's own recommendations never needed.
5. First, an automatic deployment hold suspends the release date, without requiring a human judgment call about how urgent the launch feels. Second, the result is escalated to the lab's pre-named accountable safety authority, an RSP-style Responsible Scaling Officer or an equivalent cross-functional review body, rather than being adjudicated by the product or release team. Third, an independent team, separate from whoever ran the original evaluation, attempts to reproduce and further confirm the result under harder conditions, since a single evaluation run can be a noisy artifact rather than a reliable signal. Fourth, if confirmed, the decision forks: if the result falls within a tier the policy has already defined safeguards for, the lab must implement those specific safeguards, such as structured, monitored, rate-limited API access with extra verification, before release, delaying but not cancelling it; if the result exceeds every currently defined tier, the policy instead commits the lab to pause further scaling or deployment of that capability until new, adequate safeguards are designed. Fifth, the evaluation result, confirmation process, and safeguard decision are documented, typically in a public system card, both to give future evaluations a baseline and to give outside observers evidence of whether the self-governance commitment is being honored.
6. Because the two decisions are not reversible in the same way, and reversibility is precisely the property the governance is trying to preserve. Structured access keeps the model inside the lab's own infrastructure, so if a capability turns out to be misused or riskier in practice than the pre-release evaluation suggests, the lab can tighten monitoring, add restrictions, or revoke a specific customer's access after the fact. Open-weight release forfeits that option permanently: once weights are published, every already-downloaded copy keeps running under whatever restrictions its holder chooses to honor, and no safeguard designed afterward can reach a copy the lab no longer controls. A capability tier judged safe under the assumption that access can be monitored and revoked is answering a different risk question than the same tier would face under the assumption that release is permanent and unmonitorable, which is why responsible-scaling-style policies generally treat open-weight release as requiring a materially higher confidence bar than structured access at the same evaluated capability level.
Think About It
Think about this: How would you explain ai and biosecurity: governance, policy, and institutional oversight to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.
Key Takeaways — Summary and Recap
Let us recap what we covered: the core ideas behind ai and biosecurity: governance, policy, and institutional oversight, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.