Tech & Society.

Import AI's Jacob Clark highlights two results suggesting that gains from scaling general-purpose language models are showing up in domains nobody specifically trained them for.

First, Epoch AI and METR released MirrorCode, a benchmark that measures whether AI systems can reimplement a real software program from scratch using only command-line access to the original — no source code, no web search. The model has to infer the entire program structure rather than translate line by line. Test programs range from a 61,000-line configuration language (Apple's pkl) to smaller utilities like a phylogenetic-tree parser. Claude Opus 4.7 fully reimplemented pkl and, along with GPT-5.5, successfully rebuilt the 16,000-line gotree program in multiple languages for $100-400 in inference cost. On one task, Opus 4.7 produced a working reimplementation in 14 hours for $251 that the authors estimate would take a human 2-17 weeks. Leading models from a year ago scored only ~30% on the same benchmark and could only handle much simpler programs. Some tasks remain unsolved — a Python linter (ruff), a math package, and an email-authentication library all resisted every model tested. Clark's read: this isn't just "AI got better at coding," it's evidence that models can self-orient inside an unfamiliar environment and rebuild a working model of it purely from input-output access.

Second, Anthropic's "Project Fetch" work shows the same dynamic in robotics. In August 2025, Claude Opus 4.1 couldn't help a quadruped robot do useful physical tasks at all; humans using the model were about twice as fast as those without it, but still took 181 minutes to finish a task set. By May 2026, Opus 4.7 completed nearly the entire task set autonomously in under 10 minutes — without any dedicated robotics training. Anthropic attributes this entirely to general model scaling: "these improvements... have emerged from much more general scaling." Robotics startup Sunday makes a parallel bitter-lesson argument with its ACT-2 model for home laundry-folding robots: the way to close the reliability gap in unpredictable home environments is to start with a much stronger pretrained base model and layer on small amounts of high-quality in-house data, rather than trying to engineer robustness directly. Sunday reports a 99.1% fold success rate across 778 attempts on nine garment types and plans to deploy the system, called Memo, to families this fall.

The economic angle: if general-purpose model scaling really does deliver "free" gains in adjacent domains like robotics, it could undercut years of narrow, bespoke robotics engineering investment, concentrating value capture in whoever owns the best foundation models rather than the best robot hardware or task-specific training pipelines.

A new arXiv paper identifies "Narrative Anchoring": large language models used for clinical diagnostic reasoning give different diagnoses for the exact same medical facts depending on the sociolinguistic register in which those facts are phrased — even with zero demographic markers (no race, income, or identity cues) present anywhere in the text. This differs from prior bias research that manipulates explicit identity tokens; here the only variable is how a patient's story is told.

The researchers built a dataset of 1,000 USMLE clinical vignettes, each rewritten into three sociolinguistically distinct personas while preserving the underlying clinical facts (verified by an independent auditing model that never saw the generation prompt). Across seven models spanning three architecture families, the bias was statistically significant in every model under direct prompting. Chain-of-thought reasoning and explicit debiasing instructions only partially reduced it, and apparent improvements were often confounded by an overall drop in diagnostic accuracy — meaning the model got "less biased" partly by getting worse at diagnosis generally. The team's proposed fix, NarrativeShield, structurally extracts and verifies clinical facts before diagnostic reasoning begins, cutting the bias to near-zero at a modest accuracy cost. Notably, they found that even attempting a debiasing intervention at all requires a baseline level of instruction-following ability — a non-instruction-tuned base model couldn't execute the correction regardless of prompt wording.

The human angle is direct: if deployed in real clinical decision support, this kind of register sensitivity could mean patients who describe symptoms in less "clinical"-sounding language receive systematically different diagnostic suggestions than those who don't — a new axis of inequity that has nothing to do with the patient's actual condition or explicit demographic identity, and would be invisible to standard demographic-bias audits.

A study using two safety-tuned variants of Google's Gemma-3-1B-IT model finds that combining separately fine-tuned safety capabilities through model merging doesn't preserve them evenly — one behavior can quietly overwrite another. The researchers fine-tuned one model on harm-level classification (CARES) and another on adversarial jailbreak refusal (WildJailbreak), then merged them using four standard techniques (Linear, SLERP, TIES, DARE-TIES).

Across all four merging methods, the merged models retained 81-85% of their jailbreak-refusal ability but their harm-classification accuracy collapsed to as low as 12.9%. The cause wasn't that the two safety objectives directly conflicted — the underlying weight-space "task vectors" for the two behaviors were nearly orthogonal (cosine similarity of just 0.011). Instead, the refusal fine-tune simply produced much larger weight updates than the classification fine-tune, and magnitude-sensitive merging methods default to favoring whichever update is bigger, regardless of which behavior is actually more important to preserve.

The implication for anyone deploying merged models in production: teams may believe they're combining multiple safety properties into one model, when in practice the merge can quietly collapse nuanced harm recognition into blanket, indiscriminate refusal — a failure mode that wouldn't show up unless you specifically test for both behaviors separately, and one that argues against treating "safety" as a single property that additive model-merging techniques can reliably preserve.

A new benchmark called PAUSE tests personal AI assistants in conditions closer to real deployment: managing persistent user state, respecting user-specific permissions and configurations, and sustaining long, constraint-aware interactions across multiple connected services (rather than executing one-off tool calls in a static environment). Unlike prior benchmarks that abstract away or fragment this context, PAUSE simulates realistic users interacting with the assistant over multiple turns and evaluates both open-ended service-management tasks (via semantic and behavioral metrics) and constraint-heavy tasks (via deterministic state verification).

The headline result: even state-of-the-art proprietary models fail to reach 70% task completion on scenarios that require tracking state and respecting configuration constraints over time, with consistent, interpretable failure patterns. This lands directly on the gap between the "AI assistant" marketing narrative and what's actually deployable today — coordinating a calendar, a smart-home device, and a permissions setting reliably across a conversation is still a hard, unsolved problem, not an implementation detail.

Relatedly, a separate benchmark (LayerRAG-Bench) examining agentic retrieval-augmented generation systems found a similar pattern at the infrastructure level: fixing one failure layer (like schema drift, where success jumped from 0% to over 91%) doesn't fix others, such as stale evidence, missing tool output, or denied permissions — and systems that only check whether an answer is "grounded" in retrieved evidence produce false positives even when that evidence is stale or pulled from the wrong session. Both papers point to the same lesson for anyone evaluating agentic AI products: a fix that works for one class of failure is easily mistaken for a general reliability guarantee when it isn't.

A large-scale study evaluated how well seven large language models align with human emotional perception of politically and geopolitically charged news headlines. Researchers surveyed 3,011 people, a representative sample of the UK adult population via YouGov, asking whether specific headlines evoked sympathy for a named side in a conflict, and compared human answers to those given by the models.

Alignment varied substantially by model: GPT-5.2 correlated very highly with human judgments (0.789), while Mistral Large 2512 only reached a medium correlation (0.4). Encouragingly, the leading models were broadly aligned with human judgments across demographic subgroups — age, gender, education level, prior geopolitical knowledge, and people's existing views on the conflict — though the study still found statistically significant differences between groups. The authors frame this as a distinct and often-overlooked risk category: even when a model's aggregate performance looks strong, its alignment with any given demographic or cultural group may differ in ways that matter for how fairly it represents contested framing to different audiences. As LLMs increasingly shape how people consume news and form views on conflicts, this kind of differential alignment — separate from outright factual bias — has real implications for how "neutral" AI-generated news summaries or framing actually are in practice.

Speaking on the German podcast Tech & Tales, Nicole Büttner — a partner at Merantix Capital, a European AI-focused venture fund that recently closed a new fund, and formerly a co-founder of AI implementation firm Merantix Momentum — argues that Europe's real opportunity in AI isn't in the flood of pitch decks she sees optimizing existing back-office processes (procurement, accounting, agentic workflow automation that makes something "10-20% faster"). Those are legitimate businesses, she says, but they don't excite her as an investor.

What does excite her is founders willing to rethink an entire sector from the ground up rather than incrementally improving today's version of it — her examples include rebuilding how companies purchase electricity by becoming a new kind of energy provider, rather than building a workflow tool on top of the existing purchasing process. She points to a portfolio company, Ovom, building AI-native fertility clinics as a concrete case: fertility data today is fragmented across different labs, treated as a niche "women's health" issue with little sophisticated individualized modeling (patients are largely just told to worry about fertility declining after 30 rather than getting a personalized probability estimate), and the patient journey through treatment is often isolating and emotionally difficult. She sees AI applications across three angles — unifying fragmented data collection to better estimate individual conception probability, providing emotional companionship through an often lonely treatment journey, and improving the productivity of scarce clinical staff so treatment economics work at scale. Asked whether European companies can compete globally in a heavily regulated sector like healthcare, Büttner argues it's a real opportunity precisely because the clinics she's backed have built patient consent processes that let them work within data protection rules from day one, rather than treating regulation purely as an obstacle.