The most expensive automation is the test that should never have been written. Teams burn time maintaining brittle suites that automate the wrong things — not because they lack skill, but because they automate by habit rather than by decision.
Here is the simple framework I use to decide what is worth automating. Before you write a single Playwright test, answer two questions: how often will this check run, and how painful is failure?
Free companion download
Automation Decision Checklist 2026 (PDF) · 29 KB
The two questions that decide everything
- 1
Frequency
High = it runs every sprint, regression, release or daily pipeline. Low = one-off, exploratory, rare or rapidly changing.
- 2
Risk / pain
High = failure blocks revenue, access, core workflows, compliance or critical data. Low = cosmetic, low-impact or easily spotted by hand.
Plot a check against those two axes and the right decision usually becomes obvious.
The four quadrants
- High frequency + high risk → Automate here. These are your best candidates: critical journeys that repeat often and give fast, valuable feedback.
- High frequency + low risk → Selective automation. Automate only if it saves meaningful time and is stable. Keep the suite lean.
- Low frequency + high risk → Investigate first. High impact but infrequent — consider targeted automation, monitoring, or a manual release check.
- Low frequency + low risk → Keep manual. Usually not worth the automation cost; explore or check it when needed.
Good first Playwright candidates
When you are starting out, automate the journeys that are both important and stable:
- Login and authentication.
- Checkout and payment paths.
- Account creation.
- Critical permissions and access control.
- Core create-read-update-delete (CRUD) journeys.
- Stable regression checks you run repeatedly.
Usually keep manual (or explore)
- One-off exploratory ideas.
- Rapidly changing prototypes.
- Visual nuance and subjective UX.
- Low-risk copy or cosmetic checks.
- Rare edge cases with a high setup cost.
- Anything cheaper to check manually than to maintain as code.
The pre-automation checklist
Before you automate a check, make sure you can tick all of these:
- It repeats often.
- Its failure actually matters.
- The behaviour is reasonably stable.
- The expected result is clear.
- The test can run independently.
- Automation will save more time than it costs to maintain.
Learn Playwright in the right order
Do not start with framework architecture. Write one test, make it stable, add a few genuinely valuable journeys, feel the duplication and maintenance pain for yourself — and only then introduce Page Objects, test data, Git and CI/CD. The structure means more once you have felt the problem it solves.
The rule to remember: do not automate because you can. Automate because the feedback is valuable.
Key takeaways
- Decide what to automate by frequency × risk, not by habit.
- Automate high-frequency, high-risk journeys first; keep low-value checks manual.
- Run the six-point checklist before automating anything.
- Learn Playwright by feeling the pain first, then adding structure.
Frequently asked questions
What should you automate first in test automation?
Start with checks that are both high-frequency and high-risk — login, checkout and payment, account creation, critical permissions and core CRUD journeys. These repeat often and their failure genuinely matters, so automating them gives fast, valuable feedback for the least maintenance risk.
What should not be automated?
Avoid automating one-off exploratory ideas, rapidly changing prototypes, subjective UX and visual nuance, low-risk cosmetic checks, and rare edge cases with high setup cost. If something is cheaper to check manually than to maintain as code, keep it manual.
How do I decide whether a test is worth automating?
Use a simple frequency × risk judgement, then a checklist: does it repeat often, does its failure matter, is the behaviour stable, is the expected result clear, can it run independently, and will automation save more time than it costs to maintain? If you cannot tick those, think twice.