스크레이퍼에 브라우저가 필요하지 않을 수 있습니다. 올바른 요청이 필요합니다.

The default move when a page is JavaScript-heavy is to reach for headless Chrome. Spin up Puppeteer, wait for the DOM to settle, scrape the rendered result. It works. It also costs you a couple hundred megabytes of RAM per worker, a few seconds per page, and a proxy bill that scales with how much you hate money.

Most of the time you don’t need it. The page you’re looking at is JavaScript-heavy precisely because the data arrives separately, as JSON, over an API the site’s own frontend calls. If you can call that API directly, you skip the browser, the render wait, and most of the cost. I’ve run a SHEIN scraper on Apify this way with no proxies at all, and it holds up at volume.

The data is already JSON, somewhere

Think about how a modern shop is built. The HTML shell loads, then the frontend fires XHR/fetch requests to some backend-for-frontend, gets back clean JSON, and paints the page. That JSON is the thing you want. It’s already parsed, already structured, and it doesn’t have a single CSS selector waiting to break on you.

Rendering the page to scrape the DOM is doing all that work twice: once by the browser, once by you. Going straight to the JSON cuts out the browser entirely.

How to actually find it

Open the site in a normal browser and open devtools. Then:

1. Go to the Network tab and filter to Fetch/XHR. This drops images, fonts, and scripts, and leaves the data calls. Clear it, then reload or click the thing that loads the products.
2. Watch what comes back. Click a request, look at the Preview tab. You’re hunting for a response that looks like the page’s data: a list of products, a price, an ID. When you find one, you’ve found your target.
3. Read the request that produced it. What’s the URL shape, what query params or body does it take, what headers actually matter. Most headers are noise. A few are load-bearing. You find out which by removing them one at a time.
4. Check the mobile site. This is the tip people skip. The mobile web app or the app’s own backend is often simpler, less defended, and returns flatter JSON than the desktop equivalent. If the desktop API is a pain, m. something or the mobile user-agent path frequently isn’t.
5. Reproduce it outside the browser. Right-click the request, copy as cURL, paste into a terminal. If it still returns data, you have a scraper. If it 403s, something in the headers or params was doing authentication work, and now you know where to look.

Here’s the shape of what you end up with once the browser is gone:

const res = await fetch(endpoint, {
  headers: {
    "user-agent": MOBILE_UA,
    "accept": "application/json",
    // the handful of headers that actually gate the response,
    // discovered by removing them until it breaks
  },
});

const json = await res.json();
const products = json.data.items.map((it) => ({
  id: it.goods_id,
  title: it.goods_name,
  price: it.retail_price.amount,
  currency: it.retail_price.currency,
}));

Enter fullscreen mode Exit fullscreen mode

That’s the whole thing. No render loop, no waitForSelector, no headless binary to keep patched.

The parts I won’t hand you

I’m being deliberately vague about the exact endpoints, params, and signing for any specific site, including SHEIN. Figuring those out is the actual work, and it’s a moving target that shifts when the site ships a new app version. What I’ll give you is the method. What I won’t is the finished recipe, partly because it’s my edge and partly because it’d be stale by the time you read it.

Some genuinely need a browser, and I’ll say so. Endpoints signed with a token generated by obfuscated client-side JS, or flows gated behind a real interaction, are sometimes not worth cracking. Rendering once to harvest a token, then firing cheap direct requests after, is a fair compromise. But reach for that after you’ve checked the Network tab, not before.

Rule of thumb I’d actually bet on: if a site has a mobile app, it has an API you can probably reach, and it’s probably nicer than anything you’d scrape off the desktop DOM. Start there.

If you want a worked example that runs proxyless in production, my SHEIN product scraper on Apify is built exactly this way: apify.com/native_emblem/shein-product-scraper. And the broader platform version of this lives at cartpie.com, an e-commerce product-data scraping platform with a free tier live now.