JavaScript rendering | crawler | Spatie

 SPATIE

  Crawler
==========

spatie.be/open-source

  [Docs](https://spatie.be/docs)  [Crawler](https://spatie.be/docs/crawler/v9)  Advanced-usage  JavaScript rendering

 Version   v9

 Other versions for crawler [v9](https://spatie.be/docs/crawler/v9)

- [ Introduction ](https://spatie.be/docs/crawler/v9/introduction)
- [ Installation & setup ](https://spatie.be/docs/crawler/v9/installation-setup)
- [ Support us ](https://spatie.be/docs/crawler/v9/support-us)
- [ Questions and issues ](https://spatie.be/docs/crawler/v9/questions-issues)
- [ Changelog ](https://spatie.be/docs/crawler/v9/changelog)
- [ About us ](https://spatie.be/docs/crawler/v9/about-us)

Basic usage
-----------

- [ Your first crawl ](https://spatie.be/docs/crawler/v9/basic-usage/starting-your-first-crawl)
- [ Crawl responses ](https://spatie.be/docs/crawler/v9/basic-usage/handling-crawl-responses)
- [ Using observers ](https://spatie.be/docs/crawler/v9/basic-usage/using-observers)
- [ Collecting URLs ](https://spatie.be/docs/crawler/v9/basic-usage/collecting-urls)
- [ Filtering URLs ](https://spatie.be/docs/crawler/v9/basic-usage/filtering-urls)
- [ Testing ](https://spatie.be/docs/crawler/v9/basic-usage/testing)
- [ Tracking progress ](https://spatie.be/docs/crawler/v9/basic-usage/tracking-progress)

Configuring the crawler
-----------------------

- [ Concurrency & throttling ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/crawl-behavior)
- [ Limits ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/setting-crawl-limits)
- [ Extracting resources ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/extracting-resources)
- [ Configuring requests ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/configuring-requests)
- [ Response filtering ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/handling-responses)
- [ Respecting robots.txt ](https://spatie.be/docs/crawler/v9/configuring-the-crawler/respecting-robots-txt)

Advanced usage
--------------

- [ JavaScript rendering ](https://spatie.be/docs/crawler/v9/advanced-usage/rendering-javascript)
- [ Custom link extraction ](https://spatie.be/docs/crawler/v9/advanced-usage/extracting-custom-links)
- [ Custom request handlers ](https://spatie.be/docs/crawler/v9/advanced-usage/custom-request-handlers)
- [ Crawling across requests ](https://spatie.be/docs/crawler/v9/advanced-usage/crawling-across-requests)
- [ Custom crawl queue ](https://spatie.be/docs/crawler/v9/advanced-usage/using-a-custom-crawl-queue)
- [ Graceful shutdown ](https://spatie.be/docs/crawler/v9/advanced-usage/graceful-shutdown)

 JavaScript rendering
====================

###  On this page

1. [ Configuring Browsershot ](#content-configuring-browsershot)
2. [ Using Cloudflare Browser Rendering ](#content-using-cloudflare-browser-rendering)
3. [ Custom renderers ](#content-custom-renderers)
4. [ Disabling JavaScript execution ](#content-disabling-javascript-execution)

By default, the crawler will not execute JavaScript. You can enable JavaScript rendering using the `executeJavaScript` method.

```
use Spatie\Crawler\Crawler;

Crawler::create('https://example.com')
    ->executeJavaScript()
    ->start();
```

When called without arguments, the crawler will use `BrowsershotRenderer` which requires [spatie/browsershot](https://github.com/spatie/browsershot) to be installed. If Browsershot is not installed, an exception will be thrown.

Configuring Browsershot
-----------------------------------------------------------------------------------------------------------------------------

To customize Browsershot, pass a configured instance to `BrowsershotRenderer`:

```
use Spatie\Browsershot\Browsershot;
use Spatie\Crawler\Crawler;
use Spatie\Crawler\JavaScriptRenderers\BrowsershotRenderer;

$browsershot = (new Browsershot())
    ->noSandbox()
    ->waitUntilNetworkIdle();

Crawler::create('https://example.com')
    ->executeJavaScript(new BrowsershotRenderer($browsershot))
    ->start();
```

Using Cloudflare Browser Rendering
--------------------------------------------------------------------------------------------------------------------------------------------------------------

The `CloudflareRenderer` uses a Cloudflare Browser Rendering endpoint to render JavaScript. It sends a POST request with the URL and expects a JSON response containing a `content` field with the rendered HTML.

```
use Spatie\Crawler\Crawler;
use Spatie\Crawler\JavaScriptRenderers\CloudflareRenderer;

Crawler::create('https://example.com')
    ->executeJavaScript(new CloudflareRenderer('https://your-worker.your-domain.workers.dev/render'))
    ->start();
```

You can also pass a custom Guzzle client to the renderer:

```
use GuzzleHttp\Client;
use Spatie\Crawler\JavaScriptRenderers\CloudflareRenderer;

$renderer = new CloudflareRenderer(
    'https://your-worker.your-domain.workers.dev/render',
    new Client(['timeout' => 30]),
);
```

Custom renderers
--------------------------------------------------------------------------------------------------------

You can create your own JavaScript renderer by implementing the `JavaScriptRenderer` interface:

```
use Spatie\Crawler\JavaScriptRenderers\JavaScriptRenderer;

class MyRenderer implements JavaScriptRenderer
{
    public function getRenderedHtml(string $url): string
    {
        // return the rendered HTML for the given URL
    }
}
```

Then pass it to the crawler:

```
use Spatie\Crawler\Crawler;

Crawler::create('https://example.com')
    ->executeJavaScript(new MyRenderer())
    ->start();
```

Disabling JavaScript execution
--------------------------------------------------------------------------------------------------------------------------------------------------

If you've enabled JavaScript rendering but want to disable it later in a chain:

```
use Spatie\Crawler\Crawler;

$crawler = Crawler::create('https://example.com')
    ->executeJavaScript();

$crawler->doNotExecuteJavaScript();
```