Every week a new AI tool gets launched with a demo that looks perfect. The demo was chosen by the people who made it. Your job is to find out whether the tool works on your data, your language and your deadline. You can do that in one afternoon if you keep the test small and honest.

Here is the plan I use. It works for developers and non-developers alike.

Step 1: Write down the one job

Pick one task you actually do, not a general "be more productive" goal. Examples: summarise a customer email, draft a product description in Hindi, turn meeting notes into a task list, extract fields from a PDF invoice.

Write the job in one sentence, with the input and the output. "Input: a support email. Output: a reply under 80 words in the same language, with the order number quoted." If you can't write it that way, the tool test will not tell you anything.

Step 2: Build a small test set

Collect 10 to 20 real examples from your own work. Include the messy ones: a typo-filled message, a half-English and half-Hindi request, a case with missing information.

Save them in a plain text or spreadsheet file. Don't clean them up. The tool needs to handle the mess you actually have.

For each example, write down the answer you would accept. This is your reference. Keep it short.

Step 3: Run the same test on two tools

Use the same prompt and the same examples. Don't tune the prompt for one tool. If you have to write a special prompt for a tool to work, count that effort too.

Score each output on three things:

  • Is it correct?
  • Does it need editing before you could send it?
  • Did it invent anything that is not in the input?

The third one matters most. A tool that makes things up is expensive, even if the writing is nice.

Step 4: Check the failure cases

Look at the three worst outputs. Ask whether a human reviewer would catch the error quickly. If the error is the kind nobody notices, be careful. Add a human check to the workflow before you trust it with customers.

Step 5: Decide on a small rollout

If the tool passes, use it on one week of real work with a human review. Track the time it saves and the number of corrections. If it fails, write down why. That note is useful for the next tool.

Two honest notes

Don't rely on a single benchmark or a launch video. Those measure what the vendor chose to show.

Also check the data policy before you paste customer information into any tool. Read the terms that cover your plan, not the marketing page.

What this gets you

After one afternoon you will know three things: whether the tool does your job, how much checking it needs, and what it costs you in time. That is more useful than any ranking list.

I teach these habits to students and small business owners at Aivesh Academy in Indore, and most of the value comes from the test set, not the tool.

— Devesh Sarkaar, founder of Aivesh Academy & Gharleads, Indore. More at https://deveshsarkaar.com