How to turn a no-API portal into a REST endpoint

A portal with no public API is not a portal with no API. Almost every modern public listing site is a JavaScript application talking to a JSON backend, and that backend is a real interface with real parameters, real enums and real pagination. It is simply undocumented and unsupported.

The usual response is a DOM scraper: fetch the rendered page, parse the markup, extract the fields. That works until the next redesign, which is why scrapers built this way have a maintenance cost measured in redesigns rather than in years. The alternative is to find the JSON call the page itself makes, and make that call directly. This guide describes how to do that deliberately, using a worked example, and what to verify before anyone relies on the result. The worked examples come from an AutomationAtlas field test of Supergood, a platform that generates REST endpoints from a recorded browser session.

Check the ground rules before writing any code

This step comes first because it is the only one that cannot be fixed later. Read the target's robots.txt and its terms of service, and record what they say at the time of reading.

In the worked example, the target was a Polish government body's public listing of court-ordered auctions. Its robots.txt allows the public search and listing paths and disallows only logged-in account routes, which is what made the public search fair ground and the account area out of bounds.

Two things follow. A tool that generates the endpoint for you removes essentially all of the technical friction between finding data and polling it on a schedule, and none of the legal judgement. And a rule that was permissive when you read it can change, so the check belongs in the maintenance cycle rather than in the first afternoon.

Read the bundle, not the page

The rendered page is the worst place to look for a data model, because it shows only what the current view needed. The compiled JavaScript bundle carries the whole contract, including the parts no view has ever shown you.

In practice there are three places worth opening, in this order. The network tab comes first, because the request bodies from a single real search tell you which endpoint to aim at and what its parameters are called, which is what makes everything after it searchable. The server-rendered page state, which frameworks such as Nuxt and Next embed in the HTML as a JSON blob, comes second and shows the shape of the data the page was hydrated with. The compiled bundle files come last and are where the payoff is, because a validated form control has to know every value it accepts, so the bundle carries the whole enum where the network tab showed you one member of it.

Against the auction portal this produced the entire taxonomy from two recorded searches: two main categories, eight real-estate and seventeen movable sub-category codes with their Polish labels, and sixteen provinces plus an all-provinces sentinel. Both recorded searches had been real-estate searches in a single province. Everything else came out of the bundle.

When there is no bundle to read

The step above assumes a front end that carries its own data model. A classic server-rendered application does not. There is no XHR traffic to record, no hydration blob in the HTML and no enum table in a bundle, because the server decided everything before the page was sent. What is left is rendered markup, which puts you back in the position this guide opened by criticising.

That case is worth planning for rather than discovering. AutomationAtlas tested it against a public parcel-records portal that one vendor licenses to local government, and whose own site listed more than a thousand US municipalities on the same template in September 2026. The stack is ASP.NET WebForms, with hidden viewstate and event-validation fields on every page and full-page postbacks throughout.

Three things change. The unit of work becomes the navigation rather than the call, so a recorder built around XHR and fetch traffic will capture the form POSTs and miss the document GETs between them. Check what a recording actually holds before trusting anything derived from it. The hidden state fields are part of the contract rather than noise, since a postback that omits them is rejected, so they have to be read from the preceding page rather than hardcoded. And the parse becomes the fragile part, failing in ways a JSON contract cannot.

In the worked example, a table header authored with a line break inside it, Gross on one line and Area on the next, collapsed under naive text extraction into a single token. That broke the header-to-field mapping and silently dropped two area fields from the output. That is the shape to expect on this class of target: not an error, but a field that quietly stops being populated. It argues for one check beyond the five above, which is to assert on the presence and the split of every field parsed out of markup, against a record whose values you have read off the live page yourself.

Prove the parts nobody demonstrated

An integration generalised from two examples is a hypothesis until an undemonstrated path returns real data. Test the parameter combination that was never recorded, not the one that was.

The second row is the one that matters most. A generated integration that only ever works on the exact parameters you demonstrated has memorised your examples rather than recovered the interface.

Expect the second capability to be harder than the first

Translating one call into one endpoint is the easy case. The work gets structurally harder the moment a second capability has to orchestrate the first, because that is where a generated integration meets the constraints of the platform running it.

In the worked example the second capability walked the existing search endpoint's pagination twice, once per region, aggregated price statistics for each, and computed the difference. Three things surfaced. The platform turned out to run every code block an integration carries but return only the last one's output, with no merge and no fallback, which is not in any visible documentation and had to be established by experiment. A debug probe left at the top of a block made the entire implementation unreachable dead code, caught by the platform's automated review rather than by AutomationAtlas. And the build notes claimed a code path had never run against live data for a reason that was demonstrably wrong, while the claim itself was still true.

The general lesson is that undocumented platform behaviour is discoverable by controlled experiment, and worth discovering before it costs a silent regression. Design the experiment to cost nothing: run it in whatever dry-run or replay mode the platform already offers, and make each throwaway call test exactly one proposition. Three such calls established the block-ordering rule in the worked example, in the order that both blocks run, that only the last one's output ships, and that a shared context object survives between them. Then absorb the constraint into a parameter rather than working around it. The fix here was an operation field defaulting to the behaviour existing callers already had, gating both blocks so exactly one does real work per call.

What to check before anyone relies on it

Treat a generated integration as code you now own, because you do.

Read the generated code rather than only its summary, since notes an agent writes about its own work are a best guess and not a specification. Confirm which calls are real: a replay against a recording is a regression test, not evidence that anything reached the network. Look for facet-count or telemetry calls that look like data requests and are not, since a call carrying a zero limit is usually asking how many results exist rather than asking for results. Check the failure shape, because an undocumented backend has undocumented error states and no contract that tells you what they are.

Then schedule the thing that actually decays. The wire format is more stable than the markup, but it is not permanent, and the check that catches a change cheaply is a scheduled call against a known query with a known answer.

Editor's Note: Two searches we recorded produced a schema covering 25 sub-category codes and 16 provinces, and on the second target a single line break inside a table header silently dropped two fields from the output. The Polish auction portal is a modern single-page app, and modern apps give up their data model almost by accident, which makes the method look easier than it is. The county parcel portal is the honest case: no XHR traffic, no embedded state, nothing to read but rendered markup. The method still works there, but the failure mode changes from a wrong answer to a missing field, and the check in the markup section above is the one that catches it. Budget for that target, not the flattering one. — Rafal Fila