Getting started | laravel-url-ai-transformer | Spatie SPATIE Laravel URL AI Transformer ============================= spatie.be/open-source [Docs](https://spatie.be/docs) [Laravel-url-ai-transformer](https://spatie.be/docs/laravel-url-ai-transformer/v2) Basic-usage Getting started Version v2 v1 Other versions for crawler [v2](https://spatie.be/docs/laravel-url-ai-transformer/v2) [v1](https://spatie.be/docs/laravel-url-ai-transformer/v1) Getting started - [ Introduction ](https://spatie.be/docs/laravel-url-ai-transformer/v2/introduction) - [ Support us ](https://spatie.be/docs/laravel-url-ai-transformer/v2/support-us) - [ Installation & setup ](https://spatie.be/docs/laravel-url-ai-transformer/v2/installation-setup) - [ Questions and issues ](https://spatie.be/docs/laravel-url-ai-transformer/v2/questions-issues) - [ Changelog ](https://spatie.be/docs/laravel-url-ai-transformer/v2/changelog) - [ About us ](https://spatie.be/docs/laravel-url-ai-transformer/v2/about-us) Basic usage ----------- - [ Getting started ](https://spatie.be/docs/laravel-url-ai-transformer/v2/basic-usage/getting-started) - [ Registering transformations ](https://spatie.be/docs/laravel-url-ai-transformer/v2/basic-usage/registering-transformations) - [ Writing your own transformers ](https://spatie.be/docs/laravel-url-ai-transformer/v2/basic-usage/writing-your-own-transformers) Advanced usage -------------- - [ Structured output ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/structured-output) - [ Customizing AI models ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/customizing-ai-models) - [ Customizing the stored result ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/customizing-the-result) - [ Conditional transformations ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/conditional-transformations) - [ Crawling URLs ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/crawling-urls) - [ Exploring command options ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/exploring-command-options) - [ Regenerating results ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/regenerating-results) - [ Handling failures ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/handling-failures) - [ Listening for events ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/events) - [ Using your own model ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/using-your-own-model) - [ Customizing the job ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/customizing-the-job) - [ Overriding actions ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/overriding-actions) - [ Testing transformers ](https://spatie.be/docs/laravel-url-ai-transformer/v2/advanced-usage/testing-transformers) Getting started =============== ### On this page 1. [ Running transformations ](#content-running-transformations) 2. [ What's in the database? ](#content-whats-in-the-database) 3. [ Retrieving transformation results ](#content-retrieving-transformation-results) 4. [ Scheduling transformations ](#content-scheduling-transformations) Let's build a simple example that transforms blog posts into structured data. We'll use the `LdJsonTransformer` that comes with the package to extract structured information from web pages. First, you should use the `Transform` class to register URLs to transform, and which transformer to use: ``` use Spatie\LaravelUrlAiTransformer\Support\Transform; use Spatie\LaravelUrlAiTransformer\Transformers\LdJsonTransformer; // typically, in a service provider Transform::urls( 'https://spatie.be/blog', 'https://spatie.be/open-source', 'https://spatie.be/about-us' )->usingTransformers(new LdJsonTransformer); ``` Running transformations ----------------------------------------------------------------------------------------------------------------------------- Now, you can transform the URLs by running: ``` php artisan transform-urls ``` The command fetches each URL once and then dispatches a queued job for every transformer registered for that URL. Each queued job prepares the fetched content, sends it to the configured AI, and stores the response in the `transformation_results` table. Start a queue worker to process those jobs: ``` php artisan queue:work ``` URL fetching happens synchronously while `transform-urls` is running. The AI transformations run on the queue, so retry, backoff, and queue middleware settings apply to the transformation itself. During local development, or whenever you intentionally want to wait for all transformations, use `--now` to run every transformation job synchronously: ``` php artisan transform-urls --now ``` What's in the database? ------------------------------------------------------------------------------------------------------------------------- The `transformation_results` table stores all transformation data with the following fields: - url: The URL that was transformed - type: The transformer type (e.g., 'ldJson', 'summary') - result: The AI-generated content stored as text - successfully\_completed\_at: Timestamp when the transformation completed successfully - latest\_exception\_seen\_at: Timestamp of the most recent error (if any) - latest\_exception\_message: The error message from the last failed attempt - latest\_exception\_trace: Stack trace for debugging failed transformations - created\_at: When the record was first created - updated\_at: When the record was last modified The `latest_exception` fields will be cleared when the transformation completes successfully. Retrieving transformation results ----------------------------------------------------------------------------------------------------------------------------------------------------------- Here's how you can retrieve transformation results in your application. ``` use Spatie\LaravelUrlAiTransformer\Models\TransformationResult; // Get structured data for a specific URL $ldJsonData = TransformationResult::forUrl('https://spatie.be/blog', 'ldJson'); ``` The first parameter is the URL, the second parameter is the transformer type. By default, the transformer type is the camelCased class name of the transformer without the `Transformer` suffix. You can also pass the transformer class name instead of the type. ``` $ldJsonData = TransformationResult::forUrl('https://spatie.be/blog', LdJsonTransformer::class); ``` If you need the underlying model instead of just the result, use `findForUrl()`: ``` $transformationResult = TransformationResult::findForUrl('https://spatie.be/blog', 'ldJson'); $transformationResult->successfully_completed_at; // when the transformation last completed ``` Scheduling transformations -------------------------------------------------------------------------------------------------------------------------------------- To keep your transformations up to date, schedule the command to run periodically: ``` // In routes/console.php use Illuminate\Support\Facades\Schedule; Schedule::command('transform-urls')->dailyAt('02:00'); ``` A good match? ------------- ### What we do best - All things Laravel - Custom frontend components - Building APIs - AI-powered features - Simplifying things - Clean solutions - Integrating services ### Not our cup of tea - WordPress themes - Cutting corners - Free mockups to win a job - "Just execute the briefing" In short: we'd like to be a **substantial part** of your project. [ Get in touch via email ](mailto:info@spatie.be?subject=A%20good%20match%21&body=Tell%20us%20as%20much%20as%20you%20can%20about%0A-%20your%20online%20project%0A-%20your%20planning%0A-%20your%20budget%0A-%20%E2%80%A6%0A%0AAnything%20that%20helps%20us%20to%20start%20straightforward%21)