Emergency AI Is Officially Prohibited And Unofficially Logged In

Most emergency management agencies don’t permit generative AI, according to an October 28 Grist report copublished by WIRED. The experts interviewed nevertheless describe unofficial use as widespread, including workers using personal accounts and free versions to help write messages.

We have reached the stage of technological governance where the official policy and the browser history may require separate disaster plans.

This isn’t a measured census proving that every agency has a secret chatbot. It is a warning from people who train and work with emergency managers: the technology is arriving through actual working conditions rather than the approved entrance. The government can prohibit a tool. It has not thereby supplied the time, staff or training that made the tool attractive.

Congratulations on eliminating the problem from the policy document. Unfortunately, the policy document is not the part answering three phones.

Disaster researcher Jeannette Sutton describes agencies with limited resources and personnel, trying to communicate without screwing up. Experts say humans routinely produce warnings full of jargon, missing locations or unclear instructions. This is not a contest between fallible software and an enormous national reserve of perfectly rested people who remember every training session.

The perfectly rested people are unavailable. Presumably another department has them.

AI can help. The report demonstrates a model improving a flood warning by identifying the sender, clarifying who should leave and adding safety advice and a route to updates. Some locations still need filling in. The machine has improved the sentence; somebody still needs to know which bit of the planet the sentence concerns.

That is a fairly important finishing touch. Autocomplete is not a substitute for complete.

The company behind that example, EM1, has contracts with roughly a hundred government agencies and school districts. At the time of reporting, none was using it to automate extreme-weather alerts: the work was planning and internal preparation. Meanwhile, Sarah Miller, a professor and longtime disaster-training instructor, says unofficial users are working through personal or free accounts.

So the serious version involves preparation, testing and human expertise. The improvised version is what people can get into without a procurement process. This is how public institutions risk outsourcing their technology strategy to whichever button says Continue with Google.

That button is very efficient. It has never once requested an appropriation.

There are real reasons to be careful. Emergency-management expert Laura Myers recounted an inaccurate AI-generated graphic about high winds, produced by a local manager trying to communicate the risk. Experienced humans need to evaluate the output. A novice in a hurry may not know what is wrong, and greater speed can also mean more warnings and more warning fatigue. Whether AI-generated material helps or harms public trust remains uncertain.

Which is why a ban that coexists with unacknowledged use is such a goddamn achievement. It preserves all the excitement of experimentation while making the experiment harder to supervise. The institution gets to have a policy against the thing and a practical reason not to look too closely at whether the thing is helping somebody finish the work.

I’m not blaming the exhausted emergency manager. I’m blaming the institutional fantasy that the word prohibited is a staffing level.

It isn’t. You cannot assign it the night shift. It cannot check a location, remember an evacuation route or notice that a beautifully written warning is beautifully wrong. It has excellent attendance in the handbook and absolutely no useful qualifications.

The reporting’s most sobering point is that the promising technology still needs an experienced person. The same scarce resource we were hoping the technology would let us stop worrying about. There it is again, wearing shoes and requiring a salary.

And that is the expense no free account can cancel. Buy the software, forbid the software, use the software unofficially: the person qualified to check it keeps appearing in the calculation. This is terribly inconsiderate of the person. We have tried three technology strategies and still need to fund their job.

The chatbot can help draft a warning. It cannot make the phrase adequately staffed true by putting it in a more confident font.

Perhaps that is the appeal of keeping AI unofficial. The humans are already doing too much with too little. Now even their assistance has to moonlight.