sedum
natural-language e2e Docs GitHub
now open source

AI tests already exist. You just can't run them on every PR.

Plain-English testing isn’t new. What’s different about Sedum is the economics: it’s open source, built on Jev, so running the whole suite on every PR costs dollars a month instead of thousands.

npm install -D sedum-cli
Star on GitHub Read the docs →
Or set up Sedum with your coding agent:
A Sedum run report: each step with its source, probability, duration and cost.
five sentences, one browser, one report 0:43
what we think

Six things about testing with AI.

01

Writing tests in English is a good idea. Somebody already had it.

It works, and people use it. We're not going to pretend we invented it.

02

A test you run twice a week isn't a safety net.

You want every test on every pull request. That's where the pricing breaks: these tools charge per step, and a modest suite blows through the plan.

03

Cheap has to mean cheap at volume.

Jev is fast and cheap enough to ask on every step of every run. That changes how many tests you can afford to write, and how often you can afford to run them.

04

Open source. No black box between your tests and the verdict.

The prompts, the scoring and the cache are all in the repo, so you can see what goes to the model and what comes back. It runs on your own model key, on your own machine.

05

A pass at 60% isn't a pass at 99%.

Every check comes back as a probability measured against a threshold you set, plus a second question: does the page contradict the claim? High on both is marginal, never a pass.

06

If the page moved, the report should say so.

When a cached element stops resolving, Sedum finds it again and keeps going, then grades the run DIVERGED instead of green. The test passing and the product changing are different facts.

the numbers

What a 20-person team's PR suite costs.

About a dozen engineers, one pull request a day each, pushed twice: roughly 20 runs of every test. Fifty tests, ten steps each, three of them AI steps. Priced per step, then on Jev.

the usual way

Per-step platforms

$4,875a month
$4,750 · 380,000 credits over the 10,000 included $125 · the plan
on Jev, your own key

Sedum

$91a month, worst case
$38 · low end up to $91 · worst case, cache off
a month, one linear scale
53× lessat the worst case, on the same suite: 30,000 runs a month.
Top: a per-step metered platform at its published pay-as-you-go rates: 1 credit a step, 2 for an AI step, $0.0125 a credit, 10,000 credits in the $125 plan. Its own estimator sends volumes like this to a custom Enterprise plan, so a real quote may differ. Bottom: measured with the cache off, at Jev's posted $0.042 per million input tokens (output is free). Flows on Wikipedia, Hacker News, MDN and GitHub cost $0.0013 a run; scaled to 8.4 model calls a run, $0.0030. Simple pages cost far less, and on that suite a warm cache cut model calls from 42 to 9.
the speed

Fast enough to ask on every step.

A model call per step sounds slow. On Jev it's under half a second, so a 17-step checkout finishes in 14 seconds. The same flow takes 69 on a per-step platform.

the usual way

Per-step platforms

69seconds
on Jev, your own key

Sedum

14seconds
8.5 s · browser and page 5.7 s · waiting on the model
seconds, one linear scale
4.9× fasteron the same checkout, five runs each, cache off.
One purchase flow on saucedemo.com — log in, add two items, open the cart, check out, confirm — run five times each with every cache off, on one laptop. Sedum: 13.4–14.7 s. A per-step platform's own local runner: 65–73 s. Sedum's version has 17 steps (it also opens the page and checks the confirmation); the other has 15. The model wait is the mean of five runs, across 16 calls.
what you write

You write the sentence. Sedum finds the button.

A test is a short list of plain-English steps, with no selectors. When a test needs more, write it in TypeScript and put Playwright, test data and your own helpers between the same sentences.

tests/login.test.yaml 

        

          ready
          
          
        

      

      
    

    
Same four sentences in both formats. Switch to TypeScript to see a Playwright check between them. The scores shown are an example.

  


  

    

      

        when text isn't enough
        
Four buttons say Buy. Only one is on the blue card.

        
In the page's text those buttons are identical, so
        no text model can tell which one you meant. When a click is that ambiguous, Sedum numbers
        every candidate on a screenshot and lets a Vision LLM like Gemini Flash point at the
        right one. It picks from buttons Sedum already found, never invents one.

      

      
    

  


  

    

      

        experimental · sedum run --affected
        
Changed the cart? Sedum skips the login tests.

        
Add --affected and Sedum scores each
        test by how related it is to your change. Cart and checkout tests run. Login, search and
        footer tests are skipped, and the list shows you which.

        See what your branch would run
        
Sends your Git diff and test files to Jev, 
        which scores how relevant they are. Keep running the full suite in CI.

      

      

        

          terminal
          example
        


$ sedum run --affected
Affected selection: 5/9 tests, threshold 0.1, base main.
RUN  0.9412 "tests/cart/add-to-cart.test.yaml" (model)
RUN  0.8731 "tests/checkout/guest.test.yaml" (model)
RUN  0.6120 "tests/checkout/saved-card.test.yaml" (model)
RUN  0.1840 "tests/account/orders.test.yaml" (model)
RUN  0.0310 "tests/account/address.test.yaml" (test-or-module-changed)
SKIP 0.0620 "tests/account/profile.test.yaml" (model)
SKIP 0.0210 "tests/search/filters.test.yaml" (model)
SKIP 0.0045 "tests/docs/footer.test.yaml" (model)
SKIP 0.0012 "tests/auth/login.test.yaml" (model)

        

          
RUN / SKIPWhether this test runs on this branch.

          
0.8731How related the test is to your change, from 0 (unrelated) to 1 (clearly related). 0.1 or higher runs.

          
(test-or-module-changed)You edited this test or a step it uses, so it runs whatever its score.

        

      

    

  


  

    get started
    
Try it today, or keep an eye on it.

    

      
      
    

  


  

    

      the name
      

        
Why “Sedum”? It is the plant they put on green roofs: gravel, full
        sun, nobody up there with a watering can. It lives anyway. We wanted an e2e testing suite
        that works the same way, so we took the name and the habits with it.

        

           easy to grow
           heals itself
           thrives on neglect
           grows where nothing else will