Seven tools were driven against the real apps by four agents writing the requests a person
would actually make. Almost every operation nobody had ever driven turned out to be broken,
and every one of them reported success. So the fixes divide in two: things
that now work, and — more of them — things that now say plainly that they cannot. A tool
that returns success: True for having launched an app is worse than a tool
that is missing, because the caller has no way to find out.
maps was written against a vocabulary that does not exist
Maps.app ships no scripting definition at all — nothing in the bundle,
nothing in its Info.plist — so every one of the seven operations was
AppleScript aimed at a dictionary macOS has never had. What each of them did, measured
on the previous implementation, one run each:
- search
- 2.683s
- Activated Maps, returned
success: True and the string “please manually search in the Maps app”.
- directions
- 2.228s
- Activated Maps, returned
success: True, and suggested the caller search by hand.
- listGuides
- 0.796s
- Activated Maps,
success: True, no guide data.
- createGuide
- 0.724s
- Activated Maps,
success: True, created nothing.
- save
- −2741
- An AppleScript syntax error. It had never executed a single line, for anyone, since it was written.
- pin
- −2741
- The same.
- addToGuide
- −2741
- The same.
search and directions go through MapKit now and answer
properly. Favourites and Guides are permanently impossible: no MapKit
API exists for either and there is no readable copy on disk — the container's
Maps directory is empty and the data is CloudKit-backed and opaque. So
save, listGuides, createGuide and
addToGuide report failure and hand back a durable
maps.apple.com link an assistant can file in a note instead.
listGuides in particular must never answer with an empty list: a
store that cannot be read has to stay distinguishable from a store with nothing in it.
pin does work, through that same link.
Notes cannot be edited safely, and 0 of 56 pass the gate
Notes' HTML serialiser and its HTML parser do not agree, so the body it hands
you is not a body it will take back. Notes offers no partial edit, so every
change is a whole-body rewrite, and the losses are invisible afterwards — an image is
not in the trash, it is gone.
- attachments
- 2 → 1 → 0
- Under
set body of n to (body of n) — a pure identity write, changing nothing.
- images
- 2 → 0
- On the first write, in three clean trials, returning success every time.
- links
- text only
<a href="x">t</a> goes in and <u>t</u> comes out: the anchor text survives, the URL is dropped.
- headings
- 5 of 5
- Five input forms of
<h1> were tried; every one stored back as a bold span.
- checklists
- 10 of 10
- Ten candidate markups, all stored back as a plain bullet list with every tick cleared.
So rename, replace, setBody and
append refuse any note carrying that content. On the development
store that is 0 of 56 notes — 40 carry an attachment, 36 an image, 19 a
heading, 13 a link, and 6 are password protected. That is the honest number, and it
beats an editor that works on plain notes and eats the rest. Notes this tool creates
always pass.
There is deliberately no override flag. A flag saying “yes, lose
the images” would be set by a model that has never seen the note, on behalf of a
user who has.
messages schedule cannot work, so it refuses
Messages' scripting dictionary declares three commands and none of them takes a
date. The words “schedule”, “later” and “delay” do not appear in it anywhere.
The database has columns that look like a way in until you notice that all fifteen rows
carrying one also carry a send error.
It refuses, and a check pins that it never quietly sends immediately
instead — which is the failure that would actually hurt. It stays on the tool
surface rather than being removed, because a model asked to send something at seven in
the morning, finding no such operation, is likely to reach for send.
One contact card in nine cannot be written at all
20 of 179 real cards refuse every framework write with Cocoa error
134092 — 11.2%, and stable per card rather than intermittent: 5 of 5 attempts fail on
an affected card, 0 of 5 on each control. Measured over the whole address book, one
card at a time, each change applied and then restored to its exact recorded original.
Nothing the API exposes separates the two groups. Ruled out by testing rather than by
assuming: the container, the fetch key set, staleness, unification, linked cards,
images, and the kind of change. So it cannot be pre-flighted and retrying is
pointless. The tools refuse and name the reason, including the fact that the
Contacts app can still edit the card — because the framework's own message is
“The operation couldn't be completed”, which invites a retry that will fail
forever.
The one good property, verified: the failure is total, never partial.
Every affected card was byte-identical afterwards.
Three findings about the instruments, not about the product
Each was found while trying to verify something else, and each invalidates a
verification rather than a feature — which makes them the more expensive kind. Two are
about a call that answers true for having done nothing; the third is about
an average.
A no-op save returns true and writes nothing. The first two sweeps of
the address book used a no-op — writing back a value already there — as the safe probe,
and reported zero failures across 20 real cards. executeSaveRequest
returns true for a save whose values all equal what is stored. A no-op cannot tell a
writable card from an unwritable one, so any verification built on one proves
nothing, and the two measurements it produced were worthless. That is also why
contacts rename refuses a rename that would change nothing, rather than
reporting a success it did not earn.
And executeSaveRequest returns true for an already-deleted
contact, so its success is not evidence that anything existed. Matching
happens before the save for exactly that reason.
A third of the same kind, found a day later and described above:
the mail-coverage figure this site was about to publish, 32%, was the
mean of one account at 0% and two at 100%. Every one of these three is a measurement
that agreed with itself and described nothing.
A command that never ran, and the check that was content
mail search-bodies crashed on every invocation from a
shell. The renderer takes two arguments and the command passed one, so it raised after
doing all of the work and exited 1 — not “returns empty”, never ran. It was wired an
hour before it shipped, and a model trying to build a timeline out of a mailbox filed
two of its six complaints against it.
The reason it got through is the interesting half. The parity check between the two
front ends compares operation names, and the name was right — so it was
perfectly content with a command wired to a function it could not call. There is now a
check that invokes all 24 leaf commands against a stub. It cannot
catch a wrong query or a wrong script; it catches exactly this.
Two renderer bugs went the same way inside one hour: mail coverage read
three keys the handler does not produce, so every row printed None, and
then formatted a value already in percent as a percentage and printed
10000%. A stub that invents its own payload passes both. The check
that catches them uses the handler's real key names.
Unknown is not failure
A 120-second sweep returned success: False with the messages already
marked. Timeouts now carry a distinct outcome meaning unknown, possibly
partial, and it is raised rather than returned — a key on a dictionary can be
dropped by writing if not result["success"], which is the line that
produced the false report.
It proved itself in the wild before the release shipped. A send attempted while the machine sat at
load average 321 was cut off, and reported that the outcome was unknown and that Sent
should be checked before retrying. Sent showed nothing had gone, so the retry was safe —
which is exactly the decision success: False makes impossible.
Scope is a requirement, not an optimisation
whose read status is false over a 45,000-message Gmail All Mail does not
finish. Scoped to a single mailbox, the same query is 1.6s.
So an unscoped sweep is refused immediately, naming the mailboxes on this machine that
you could scope it to. Offering a call that cannot work, and answering it two minutes
later with an outcome nobody can report, is worse than saying no.
A verification here was wrong before it was right: the sent count was compared six
seconds after a send and showed nothing, and fifteen seconds later it had moved. A check
whose timing is wrong reports the same thing as a failure.
Two control characters cannot be stored, and it says so
U+001E and U+001F are how a refusal marks a parameter name, so
that one sentence written once prints as --delete-notes at a shell and
delete_notes to a model. Text arriving with them in it gets the same
treatment. Measured: a reminder named shop ␞account␟ now was stored
verbatim by EventKit and came back out as
Created reminder 'shop --account now' — a flag this tool never offered,
written by whoever typed the reminder.
They are now removed from every argument of every operation before anything is stored or
matched. That was already happening on the AppleScript routes, as a side effect of there
being no literal form for a control character — which is why notes and mail were
incidentally safe and the EventKit, Contacts and MapKit writes were not.
The second half is that it is no longer silent. Storing text that differs from the text
you sent and reporting plain success is the same fault as a truncated read that does not
mention it: the write goes through, and the answer carries a count of what was removed
and from which parameter — on stderr for a shell, in
removed_control_characters under --json, and appended to the
text an MCP client receives. Nothing else is touched; quotes, backslashes, emoji and
bidirectional marks are stored exactly as sent.
Observed, and not a defect here
At that same load average of 321, every Apple Events path became unusable — Mail sat at
0% CPU in a runnable state, starved rather than broken — while the reads that go
straight to the local index were unaffected, answering in
0.07s to 0.49s throughout.
Which is the three-routes picture from the top of this page,
showing up as latency instead of as a permission dialog.