How to trust an AI agent with work you can't watch

Key points

• Run a new job once while you watch, starting from only its saved instruction. A 3am run won't have your chat to lean on.
• Give each customer or ticket a task of its own, so an agent remembers them without mixing them up.
• Have the morning count come from the record, not from the AI's own account of its night.
• Decide in advance what may wake you. Everything else can wait and arrive together.
• Let a long job look up its exact facts, instead of trusting its own summary.

You trust an AI agent with unwatched work the way you'd trust a new colleague. You do the job together once, keep their work easy to check, ask for a straight account each morning, and agree what's worth waking you for. nox.cool now builds each of these into how Nox, its AI agent, works: a watched first run, one task per customer, a counted morning look and quiet hours. Here's the reasoning, and where each one stops.

#Why is unwatched work the hard part?

Most of what goes wrong with an agent doesn't happen in front of you. When you're in the chat, you correct it as you go, often without noticing you've done so. When it runs at 3am, nobody corrects anything, and a small misunderstanding can repeat every night until you spot it.

A few problems come up again and again:

• The instruction leaned on the chat. You said "my accounts" and the chat knew which. The saved routine doesn't.
• The details got mixed. One task handles every customer's messages, so one person's order number ends up in a reply to someone else.
• You don't know what happened. You wake up to a long list, or to silence, and neither tells you whether anything needs you.
• The notices come at the wrong time. Either they wake you for nothing, or you turn them off and miss the one that mattered.
• The facts went soft. A long job summarises its own work to keep going, and an exact figure becomes a rough one.

None of these is about an agent being unintelligent. They come from the gap between what you meant and what it has in front of it when you're gone. So that's what we worked on.

#What should you do before an agent runs alone?

Do the job once together, and start that run from nothing but the saved instruction.

The second half matters more than it sounds. If you test a routine in the chat where you described it, the test can pass because of everything the chat already knows. The 3am run won't have any of that.

Here's how it goes on nox.cool. Say "every morning, sum up my inbox". Nox asks for your OK before saving a routine, and the card has one extra line, ticked by default: Run it once now, so you can see it.

Approve it, and the run starts beside your chat as a card you can open and follow. It ends with the part you most need: what it had to guess, and whether that's right.

That guess is the kind of thing that would otherwise stay wrong for weeks. Now you answer in ordinary words, such as "leave out newsletters from now on", and Nox proposes the change, showing exactly what would differ in the instruction.

The instruction is all that later runs read, so that's what gets fixed. A routine that has never run also says so on the Routines page, with a link to run it.

Where it stops: a first run isn't a rehearsal. It does the real work, and a step that sends something still stops for your OK, unless you've already set a rule that allows it. If you'd rather not run it, take the tick off.

#How do you stop an agent mixing up your customers?

Give each customer or ticket a task of its own.

Say a trigger answers your support email. You have two old choices, and both are poor. A new task each time means the agent has forgotten the customer by their second message. One task for everything means it remembers all of them at once, in the same place.

A nox.cool trigger now has a third way, under Routines, then Triggers, then Each time:

Follow-ups from the same sender or ticket carry on in the same task, which remembers them, and each task stays apart from the others. It works for Gmail, Nox's own address, GitHub, Linear, Sentry, PagerDuty, Slack and Shopify, and for a plain web address, where you can name the field to go by, such as customer.email. You can also ask for it when Nox sets up the trigger.

A task that remembers someone is worth impersonating, so an email whose sender the mail check can't confirm gets a task of its own. It can't walk into a real customer's history by putting their address in the From line.

Two more things make this checkable. Test run shows which task each recent event would have gone in, before anything real happens. And when Nox sets up a trigger from a task or chat, it comes back three days later to tell you how it went: how often it started, what was skipped and why, and what it would change. It changes nothing unless you say so.

Where it stops: the daily cap belongs to the whole trigger, so one noisy sender can use it all up.

#How do you know what happened overnight?

Ask for a count that comes from the record, not from the AI's memory of its night.

An AI asked to summarise its own work will write something plausible. That's fine for prose and not good enough for a number you act on. So the line at the top of Nox's morning look isn't written by the AI at all. nox.cool's server counts it each time you open the home screen, from the same list your Inbox uses, so the two agree.

It's "overnight" in the morning and "while you were away" after noon. The count goes down as you deal with things. Press the arrow and you get who asked and when, what went wrong, and what finished, with a line from each.

Questions can be answered right there. OKs can't, on purpose: they open the task, because something you're approving should be read in full. The notice you get each morning, on your phone, Telegram or Slack, carries the same words.

The morning look also stops repeating itself. Something it has told you about won't come back for 60 days, and it's reminded what it said in the last five briefs, because a daily note that says the same things stops being read.

Where it stops: routines still compare themselves only with their last report, so a slow drift over weeks can go unmentioned.

#What should an agent be allowed to wake you for?

Very little, and you should decide that in advance.

Quiet hours in nox.cool are in Settings, under This device, or you can ask Nox. They're off until you turn them on, and then they run from 22:00 to 7:00 in your own timezone, or whichever whole hours you pick. The hours are worked out afresh each time, so a clock change can't move them.

During those hours, notices wait. When they end, one that waited arrives as it was, and several arrive as one notice that opens your home screen. An OK that has since been dealt with is left out. Work doesn't stop, only the notices do.

We left it off by default for a plain reason: if your account has the wrong timezone, quiet hours would silence your day instead of your night. If the check itself has any trouble, the notice goes out. A notice arriving at a bad time is better than one lost.

Where it stops: Telegram text messages arrive without a sound, but a voice memo or file still makes one. Slack and Teams follow their own do not disturb.

#How does a long job keep its facts straight?

It stops relying on its own summary for exact details.

Any long-running agent eventually summarises its earlier work to make room. A summary keeps the gist, and that's where an invoice number can go wrong without anyone noticing. Two changes help.

First, Nox now summarises later: at about 130 screenshots' worth of work, rather than 33. Second, the summary is no longer the only copy. Nox can search the task's own stored record, including what its tools returned, even from before the summary. A number, path or reference has to match exactly, so asking for TX-4417-C will never find TX-4417-B. It uses no AI, and it took under half a second in our tests. Nox is told to look there rather than guess or redo the work.

Where it stops: we haven't yet measured how real long tasks cost and behave either side of the later summary. Later summaries could cost more or less, and it's a setting we can move back. And this search covers the task it's in, not your other tasks.

There's a related fix for long chats. An app you connect or change now reaches a chat that's already open, from your next message. And if one of an app's tools stops being read-only, it now asks first, where before an open chat could go on treating it as safe.

#What should you ask of any AI agent?

These are fair questions before you hand any agent work you won't see, ours included:

You don't have to watch everything. You have to see the first run, read a straight count in the morning, and know the important steps still stop for your OK. If you're moving from another assistant, Bring your setup in nox.cool's Settings gives you a message to paste into it, and your routines arrive paused, so you can run each one once before you trust it. What's new in this update has the steps for each of these.

#Questions and answers

How do I know an AI agent will do the right thing when I'm not there?

You can't know for certain, so check the instruction once while you can still see it. Run it from the saved words alone, see what it had to guess, and fix the instruction. On nox.cool that's a tick on the routine's OK card, on by default.

Is a first run a safe rehearsal?

No, it's a real run. A step that sends something still stops for your OK, unless you've set a rule that allows it. That's the same as any other run, which is the point of watching one.

Can an AI agent remember each customer without mixing them up?

It can if each customer has a separate task. Set a nox.cool trigger to one task per sender or conversation, and each follow-up carries on in its own task, with no sight of anyone else's.

Will quiet hours make me miss something urgent?

Security and money problems and calls you asked for still come through. Everything else waits. They're off by default, because a wrong timezone would silence your daytime.

Do summaries make long AI tasks forget things?

They can. A summary keeps the gist and may lose an exact number. Nox keeps the summary, and can also search the task's own record, where a number or reference has to match exactly.

Where can I try nox.cool?

It's part of nox.cool's early access, which is by invite for now.