Modern web applications need reliable content guardrails. User-generated content invites spam, toxic language, and policy violations. Building robust safety filters used to require complex external classification pipelines or brittle regex matching. Today, as a PHP developer leveraging a modern php web framework, you can integrate intelligent language models directly into your backend architecture.
The Laravel ecosystem provides first-class php developer tools to streamline artificial intelligence integration. By combining the Laravel AI SDK with structured outputs, you can build a production-ready REST API endpoint that classifies toxicity, detects spam, and returns strict JSON payloads. This guide walks through building a complete content moderation API from scratch.
Setting Up the Foundation
Every robust API requires persistent storage to log moderation decisions, review flagged submissions, and audit user activity. You need a dedicated database migration and an Eloquent model to store incoming payloads and classification results.
Run your migration to create the moderation logs table. The table stores the raw content, the classification status, severity scores, and specific violation categories.
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::create('moderation_logs', function (Blueprint $table) {
$table->id();
$table->text('content');
$table->boolean('allowed')->default(false);
$table->string('severity')->default('none');
$table->json('categories')->nullable();
$table->text('reason')->nullable();
$table->timestamps();
});
}
public function down(): void
{
Schema::dropIfExists('moderation_logs');
}
};
Next, define the corresponding Eloquent model to manage database interactions cleanly.
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
class ModerationLog extends Model
{
protected $fillable = [
'content',
'allowed',
'severity',
'categories',
'reason',
];
protected $casts = [
'allowed' => 'boolean',
'categories' => 'array',
];
}
Creating the AI Agent with Structured Output
The core of your moderation service is an AI agent. The Laravel AI SDK allows you to define specialized agents that interact with providers like OpenAI or Anthropic under the hood. To ensure predictable downstream processing, you must enforce structured JSON output using the HasStructuredOutput contract.
First, define the schema DTO that represents the expected classification response.
namespace App\AI\Schemas;
use Laravel\Ai\Contracts\HasStructuredOutput;
class ModerationResult implements HasStructuredOutput
{
public bool $allowed;
public string $severity;
public array $categories;
public string $reason;
public static function schema(): array
{
return [
'type' => 'object',
'properties' => [
'allowed' => ['type' => 'boolean'],
'severity' => [
'type' => 'string',
'enum' => ['none', 'low', 'medium', 'high'],
],
'categories' => [
'type' => 'array',
'items' => ['type' => 'string'],
],
'reason' => ['type' => 'string'],
],
'required' => ['allowed', 'severity', 'categories', 'reason'],
];
}
}
Now, create the agent class. You can switch between providers like OpenAI (gpt-4o) or Anthropic (claude-3-5-sonnet) by adjusting the configuration properties.
namespace App\AI\Agents;
use App\AI\Schemas\ModerationResult;
use Laravel\Ai\Agents\Agent;
class ContentModeratorAgent extends Agent
{
protected string $provider = 'openai';
protected string $model = 'gpt-4o';
protected string $structuredOutput = ModerationResult::class;
public function instructions(): string
{
return <<<EOT
You are an enterprise content moderation system.
Evaluate the provided user text against standard safety policies:
- Flag hate speech, harassment, explicit sexual content, self-harm, severe violence, and blatant spam.
- Return a structured JSON response matching the required schema.
- Keep reasons concise and objective.
EOT;
}
}
For teams preferring Anthropic, simply change $provider = 'anthropic'; and $model = 'claude-3-5-sonnet';. The SDK normalizes underlying provider API calls seamlessly.
Building the REST API Endpoint
When you build rest api with php, clean separation of concerns keeps your controllers lightweight. Create a service class to encapsulate the AI agent execution.
namespace App\Services;
use App\AI\Agents\ContentModeratorAgent;
use App\AI\Schemas\ModerationResult;
use Laravel\Ai\Facades\Ai;
class ContentModerationService
{
public function moderate(string $content): ModerationResult
{
/** @var ModerationResult $result */
return Ai::agent(ContentModeratorAgent::class)->prompt([
'content' => $content,
]);
}
}
Next, wire the service into an API controller. Validate incoming requests, run the moderation check, log the outcome, and return a clean JSON response.
namespace App\Http\Controllers\Api;
use App\Http\Controllers\Controller;
use App\Models\ModerationLog;
use App\Services\ContentModerationService;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;
class ModerationController extends Controller
{
public function store(Request $request, ContentModerationService $service): JsonResponse
{
$validated = $request->validate([
'content' => ['required', 'string', 'max:5000'],
]);
$result = $service->moderate($validated['content']);
$log = ModerationLog::create([
'content' => $validated['content'],
'allowed' => $result->allowed,
'severity' => $result->severity,
'categories' => $result->categories,
'reason' => $result->reason,
]);
if (! $result->allowed || in_array($result->severity, ['medium', 'high'], true)) {
return response()->json([
'status' => 'rejected',
'reason' => $result->reason,
], 422);
}
return response()->json([
'status' => 'approved',
'id' => $log->id,
], 200);
}
}
Register your endpoint in routes/api.php with built-in API rate limiting to prevent abuse.
use App\Http\Controllers\Api\ModerationController;
use Illuminate\Support\Facades\Route;
Route::middleware('throttle:60,1')->post('/v1/moderate', [ModerationController::class, 'store']);
Handling Batch Processing and Queues
High-volume applications cannot evaluate large comment archives or multi-item uploads synchronously. Offload heavy moderation tasks to background queue workers using Laravel's robust queue system.
Create a queued job to process bulk content asynchronously.
namespace App\Jobs;
use App\Models\ModerationLog;
use App\Services\ContentModerationService;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
class ProcessBatchModeration implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public function __construct(public array $items) {}
public function handle(ContentModerationService $service): void
{
foreach ($this->items as $content) {
$result = $service->moderate($content);
ModerationLog::create([
'content' => $content,
'allowed' => $result->allowed,
'severity' => $result->severity,
'categories' => $result->categories,
'reason' => $result->reason,
]);
}
}
}
Queue jobs process smoothly in the background while keeping your API response times instantaneous.
Testing Your AI Agent
Testing AI integrations without incurring real API costs or external network calls is essential for CI/CD pipelines. The Laravel AI SDK provides Agent::fake() for robust unit testing.
namespace Tests\Feature;
use App\AI\Agents\ContentModeratorAgent;
use App\AI\Schemas\ModerationResult;
use Tests\TestCase;
class ModerationApiTest extends TestCase
{
public function test_api_blocks_toxic_content(): void
{
ContentModeratorAgent::fake([
'allowed' => false,
'severity' => 'high',
'categories' => ['hate-speech'],
'reason' => 'Violates community guidelines.',
]);
$response = $this->postJson('/api/v1/moderate', [
'content' => 'Sample toxic test string',
]);
$response->assertStatus(422)
->assertJson(['status' => 'rejected']);
}
}
Your test suites remain fast, deterministic, and isolated.
Conclusion
Building automated guardrails no longer requires complex machine learning infrastructure. By leveraging a reliable php web framework, powerful php developer tools, and the Laravel AI SDK, you can ship production-ready APIs with structured JSON classification in minutes.
Explore the official Laravel Documentation to discover more ways to integrate intelligent automation into your applications. We would love to hear how you build and secure your next project.