commentdraft: replying in public on a client's behalf

The job arrived the way freelance work does: a client with a real product, a real price, and more comments under their posts than they could keep up with. The obvious build is a loop that reads the comments, sends each one to a model, and posts what comes back, and it takes about an afternoon. It is also wrong in four ways, and every one of them sends the bill to the client rather than to the person who wrote the loop.

What the job needed is commentdraft, a command line tool on PyPI under Apache-2.0, generalized from that job once its constraints turned out to have nothing to do with the client in particular. It reads the comments on a creator's own posts, decides for each one whether to reply, skip, or escalate to a person, and drafts the reply from one source document the operator supplies. A person approves every draft, one keystroke per reply, immediately before that one reply is sent. The case study is the same system with more screenshots and less argument.

A wrong price is a bill somebody else pays

The first failure named the requirements. A public reply under the client's own post that quotes a price that changed last month, or promises a delivery window the product does not offer, does not crash and raises no exception. It is a wrong number, in public, under their name, read by the one audience they cannot afford to mislead. The client pays.

So the drafting rule is absolute rather than encouraged: a draft may state as fact only what the operator's source document states, and a question that document does not answer is never an invitation to be helpful. It becomes an escalation, no draft, routed to the person who can actually answer it. Declining to answer is an output.

There is no retrieval step behind that rule, and the absence is a decision. The whole document goes into the prompt, once, byte for byte. Retrieval exists to fit a corpus that does not fit a context window, and it buys that at the price of a new failure mode: the retriever misses the relevant passage, the model answers anyway, and the answer is confident and wrong. At this size a retriever would introduce the exact failure the tool exists to prevent. While the document fits the window there is nothing to miss, and when a document outgrows the window the tool says the design is wrong for it and does not truncate quietly.

Approval is structural, because a setting eventually gets turned on

The second failure is that the afternoon build posts. Every platform has rules about replying with software, almost nobody reads them, and at least one of them decides the design outright. YouTube's API Services Developer Policies require that the user "expressly consent to those actions prior to their actual execution." Read narrowly, the way a policy gets read when a client's channel is the one at stake, that rules out a batch approval, a default, and any setting that stands in for a person. What it leaves is per-reply human approval, which on YouTube is not a product preference. There is no other compliant design.

So there is no --yes, no --all, and no config key that changes it. Not defaulted off; absent, and kept absent by a test that walks the config schema, which is a frozen allowlist, so a key of any name that could stand in for a keystroke fails the build until somebody writes it down. Publishing thirty replies costs thirty keystrokes, on purpose.

Typeahead is not consent

Then I pointed three reviewers at that gate and asked them to break it. One of them did, in a way I would not have found. Start the tool under a pseudo-terminal. Before it has printed a single character, write five approvals into the terminal. Then wait.

LEAD = 1.0                    # the child sleeps this long before rendering
PAYLOAD = b"y\ny\ny\ny\ny\n"    # five approvals, delivered during the sleep
ROWS = 5

pid, master = pty.fork()
if pid == 0:
    os.execv(sys.executable, [sys.executable, driver, ...])

time.sleep(0.3)
os.write(master, PAYLOAD)     # nothing has been drawn yet

Five replies published. Not one of them was on the screen when the key that approved it arrived. No pipe, no flag, no configuration, no monkeypatching, no edit to the source. On a real terminal, in front of a real person, that sequence is a single paste.

The prompt was reading with input(). That reads a line from the terminal's input queue, and the line discipline has been filling that queue since long before the program asked. Everything typed or pasted while the reply was still being written to the screen is already sitting there, waiting.

The gate proved a key was pressed, and it proved the send happened inside the branch that key selects. It never proved the key arrived after the reply was readable, and only that last part makes it consent. Every confirmation prompt I have written has this shape. Delete 400 files? [y/N]. Deploy to production? [y/N]. If the user was typing while the tool was still computing what to warn them about, the answer was in the buffer before the question reached the screen.

_line(stream, PROMPT)                                  # 1. print the prompt
_flush(stream)                                         # 2. push it to the screen

descriptor = sys.stdin.fileno()
saved = termios.tcgetattr(descriptor)
try:
    termios.tcflush(descriptor, termios.TCIFLUSH)      # 3. discard what was typed before
    tty.setcbreak(descriptor)                          # 4. no line discipline, no Enter
    pressed = os.read(descriptor, 1)                   # 5. exactly one byte
finally:
    termios.tcsetattr(descriptor, termios.TCSADRAIN, saved)

Four lines, and the order carries all of it. Discard the queue at the moment the prompt becomes readable, then read one byte after that. Anything typed before the reader could have seen the reply is gone. cbreak earns its place twice: a key needs no Enter behind it, so it also cannot be joined to the key behind it, and a held-down key walks nothing.

The replacement quietly loses two things. input() flushes stdout before it reads and a hand-rolled reader does not, so under block buffering, which is what a piped run looks like, the prompt sits in a buffer while the program waits for a key and the reviewer approves a blank screen. The flush is step two for that reason. And the terminal has to be restored in a finally, not on the happy path, because if anything below raises or the operator presses Ctrl-C, the alternative is handing them back a shell with no echo and no line editing.

A bug about terminals has to be tested against a real terminal, so the test forks a pty, writes the payload during the lead time, and asserts that nothing was published. The first version of that test passed against the broken code. The child was a fork of the pytest process, pytest imports readline, and a forked child inherits the hook readline installs under input(). That hook handles the terminal itself, and it hid the entire behavior I was trying to catch. The fix was to launch the child as a fresh interpreter under os.execv, which is also what an operator's own process looks like.

# before
ISATTY True   RC 0   SENDS 5  ['r1', 'r2', 'r3', 'r4', 'r5']

# after
ISATTY True   SENDS 0  []
LAST ROW SHOWN ['[ 1 / 5 ]  video-site  r1 on "a clip"']

There is no RC line in the second one because the queue is still holding, waiting for a key that has to arrive after the reply was on the screen.

I had tested that the gate could not be bypassed by a flag, by a config key, by calling the function directly, or by piping yes into it. All of those were closed. I had not tested the one property the gate exists to guarantee, because I had not noticed it was a separate property from having pressed the key. Proving a keystroke happened is easy, and I had done it thoroughly.

The text you receive is not the text the person wrote

The third failure is quieter, and it belongs to no single platform. I read the comment documentation of eight platforms end to end, and on three of them the value that comes back is not the one it appears to be. None of the three says so at the endpoint that returns it. Every page quoted below was read on 2026-08-01 and can be re-read today without an account.

A YouTube comment carries two text fields. snippet.textOriginal is documented on the comments resource page as "The original, raw text of the comment as it was initially posted or last updated. The original text is only returned to the authenticated user if they are the comment's author." Nobody moderating their own video is the author of the comments under it, so on every one of them the field holding the typed words is withheld, and the one account exempt from the rule is the account that already knows what it wrote.

What arrives instead is snippet.textDisplay, described on the same page: "Even the plain text may differ from the original comment text. For example, it may replace video links with video titles." A viewer pastes a link and a title comes back. Quote that comment back at them and you publish words they never typed, and a link blocklist matched against that string never fires, because the string that held the link now holds English. A second alteration stacks on the first: textFormat defaults to html on the read endpoints, so a reader that does not explicitly ask for plainText gets markup as well.

Reddit writes down exactly what it does, on the API overview page, not at any endpoint that returns a comment: "For legacy reasons, all JSON response bodies currently have <, >, and & replaced with <, >, and &, respectively. If you wish to opt out of this behaviour, add a raw_json=1 parameter to your request." So Tom & Jerry reaches the model as Tom & Jerry. Then the trap closes a second time. Any page that renders strangers' text has to escape its output, which is the defense against script injection, so a correctly built page escapes the already-escaped string and shows a person Tom & Jerry, rendered faithfully, as if that is what got typed.

Every component did its job, and the text on the screen is still text nobody wrote.

The Instagram case is not comment text at all, and it fails the same way. An Instagram comment webhook names the account it belongs to in the payload's entry.id, so a handler's first job is to know its own account's id, and the obvious call is GET /me?fields=id. On the Instagram Login route, the get-started page carries a field table with two entries a reader skims past. id is "The app user's app-scoped ID". user_id is "The Instagram professional acount ID, <IG_ID>, for your app user. This ID is value of the id field received in webhook notifications for this account." The typo and the missing word are Meta's, on a page stamped 2024-12-02.

GET https://graph.instagram.com/v26.0/me?fields=id       # the app-scoped id: matches nothing
GET https://graph.instagram.com/v26.0/me?fields=user_id  # the id webhook payloads carry

The app-scoped id is well formed, stable, and genuinely yours. It also matches no entry.id in any webhook payload and addresses nothing on /{ig-id}/media. What it builds is a handler that returns 200 to Meta forever without recognizing a single notification as its own, next to a media listing that comes back empty instead of refused. Nothing raises. Both dashboards stay green. I am not reporting that one from a safe distance: my own research had the wrong field, and it survived until a second pass pulled Meta's raw field table rather than a summary of it. That pass found twelve errors in the research behind this one platform, and this was the expensive one, because it fails by producing nothing.

No test suite catches any of the three, because every altered value is legal. The escaped string is valid JSON holding a plausible sentence. The rewritten text is a plausible comment. The wrong id has the right shape and came from the right host with a 200. Catching them means comparing the value inside your system against the person's screen, and no harness has the person's screen. The person best placed to notice is outside the system entirely: a commenter reading their own words, misquoted back at them by an account they trusted.

Meta documents the same call as reply and as edit

The fourth failure is the one that could have destroyed something that was not mine to destroy. Meta documents one Graph API call as both "reply to this comment" and "edit this comment." Both readings are published, and they cannot both be right. Pick the wrong one and every approved reply silently overwrites the customer's own comment with the client's words: not a failed reply, which you would notice, but a customer's comment replaced in place, which you would not, until somebody complains.

I could not settle it from the documentation, and I was not going to settle it experimentally on somebody's Page. The connector proves the outcome on every write. The id the platform returns must differ from the id that was posted to, and the reply read back must carry the right parent, the comment it was answering. A read-back that cannot be performed ends the whole run rather than the row, because a write path that edits comments will edit the next one too.

created = _call("POST", _url(f"{parent}/comments", token), {"message": text})
published = _identifier(created.get("id"))
if not published:
    raise _unusable(parent, token, text)
if _same_comment(published, parent):
    raise ReplyInvariantError(_overwritten(parent), parent)
seen = _verify(published, parent, token)   # reads back id, parent{id}, message
_confirm(published, parent, token, text, seen)

The check is permanent, not a placeholder until Meta's pages agree with each other, and its two failure directions are deliberately not symmetric. A false positive halts a run over an overwrite that did not happen, which costs a re-run. A false negative destroys customers' comments quietly. The comparison is built to fail toward the first.

That connector, for Facebook Pages, is the only one. Built and tested against fakes, it has never been run against a live Page, and the other seven platforms have guides instead of code.

Eight guides, because the tutorials name the wrong obstacle

Underneath all four failures sits the question a client asks first, which is what it takes to connect to their platform at all. Answering it honestly, eight times, turned out to be most of the work. For each platform it meant reading the primary sources end to end, the developer policies, the API reference, the quota tables, the pricing pages, because the tutorials answering the same question are wrong, and wrong in a consistent direction. I went looking for these guides before writing them, and they did not exist.

X is the cleanest example. The received wisdom is that the barrier is money, and the $100, $200 and $5,000 figures still dominate the search results. Those are prices for a product X stopped selling on 2026-02-06; at this tool's shape of workload, the metered replacement bills about three dollars a month. What replaced the money is a requirement for prior written approval from X before replies written by software are deployed, with no published turnaround, no queue position, and no appeal.

TikTok is repeated everywhere as having no comment API. It has two documented comment endpoints, on business-api.tiktok.com, a different product line from the developers.tiktok.com portal every tutorial means. What TikTok refuses is not the feature but the applicant: it does not onboard individual developers, and says so on its registration page.

That pattern held across all eight. Where the received wisdom names an obstacle, the named obstacle is out of date or standing in front of a different one, so the section worth reading on each guide is the one headed "What is still unknown." The eight guides run to 7,036 lines, against 5,612 lines of Python in the package. Line counts measure typing, not truth, but the ratio says where the work went, and every endpoint, scope string, quota number and policy clause in them carries the URL it came from and the date it was read. The short version, what stands between an operator and a first working call on each platform, is at /commentdraft.

The sticker prices predicted a gap of 8; the run measured 27

Cost is where this design is easiest to doubt, because the prompt carries the whole document on every call. The model comparison therefore ships as a subcommand instead of a slide: commentdraft bakeoff runs the same comments through the default model and every challenger in the config and writes one CSV per model, and a --blind flag on the review command turns those CSVs into one page with the sources hidden behind letters and the key returned separately. One run is published in docs/bakeoff.md: thirty comments against the fictional field guide the repository ships, on 2026-08-01. One comment is empty and is decided locally as a skip with no call and no cost, so each model billed twenty-nine calls.

The arithmetic is short. The cheap entry's input rate is 8.6 times the default's and its output rate 7.9 times, so sticker prices predict a gap near 8, not 27. The rest is the prompt cache. The prefix, 4,112 tokens holding the voice rules, the worked examples, the output contract and the entire source document, dominates the bill on every call, against a user message holding one comment and a reply of a sentence or two. The default's route served that prefix from cache on 28 of its 29 calls and billed it at the cached rate; the other two billed it at full input rate on all 29, because neither served it from a cache at all. Here the no-retrieval decision arrives as a measurement: the prefix is assembled once and kept byte-identical across a run so a provider can cache it, and on this run that property was worth more than the difference in sticker price between all three routes.

The same run reports the results that do not flatter it, at the same size. The example config sets an alarm threshold on how often a reply mentions the product, and two of the three models went over it. The report line, identical for primary and cheap:

plugs: 13/16 replies contain a configured plug marker OVER plug_cap 0.75

Nothing stopped, nothing was rewritten, and no row was dropped, because the cap is an alarm and never a limiter; what holds the rate down is what the operator wrote in their voice file. The default model's report also named, row by row, the thirteen replies that closed on the same pointer, the six that opened on the same word, and the three that opened on the price, a repetition no per-comment call can prevent, since no call knows how any other call ended.

The report does not catch everything, and the screenshot near the top of this post carries one it missed. Row 22 is a question in Polish. The voice file's first rule is to reply in the language of the comment, and the default model answered it in English. The two cheaper models both answered in Polish. So the route the cost table favours is the one that broke the rule, on the only row where the rule was tested, and no threshold in the report was watching for it. A report catches what somebody thought to count, and the row it misses is the row you find later by reading.

What survived the job

The client-specific parts are gone. The product, the language, the source document and the credentials all stayed behind. What survived is the shape the constraints forced, and those constraints turned out not to be about the client at all: anyone with a price and an audience has a document that is true, questions it does not answer, platforms with rules about software that replies, and comment text the platform already touched on its way in.

The general version is on PyPI as pip install commentdraft, Apache-2.0, with one runtime dependency, the OpenAI client pointed at whatever compatible gateway the config names, and a suite of 704 tests that runs offline with no API key. One thing to know before you install it: the release on PyPI is the drafting half. It reads a CSV, writes the page of drafts, and exits. The connector and the approval gate that most of this post is about are in the repository and not yet in a release, so pip install today gets the part that cannot send anything at all.

Its claims are written to be checked rather than believed. The review page above is the tool's own output; the run behind the table is written up in the repository with the command that reproduces it; and the README is not allowed to say bot, auto-reply, engagement, or growth, because a test fails the build on each of those words. Every one of them would claim something the code does not do.

If you are about to build the afternoon version for a client of your own, read the guide for their platform before you write any code, then go and paste five characters into whatever confirmation prompt you already trust. The loop is the easy part, and it was never the part they were paying for.