marshmallow / laravel-database-sync
Sync your production database in to your local database to start developing with production data
Requires
- php: ^8.2
- laravel/framework: ^10.0|^11.0|^12.0
Requires (Dev)
- orchestra/testbench: ^8.0|^9.0|^10.0
- pestphp/pest: ^2.0|^3.0
- pestphp/pest-plugin-laravel: ^2.0|^3.1
This package is auto-updated.
Last update: 2025-06-25 09:28:18 UTC
README
A powerful Laravel package that enables seamless synchronization of data from a remote database to your local development environment.
Table of Contents
Requirements
- PHP ^8.2
- Laravel ^10.0|^11.0|^12.0
Installation
You can install the package via composer:
composer require marshmallow/laravel-database-sync --dev
Configuration
Publish the configuration file:
php artisan vendor:publish --tag="database-sync-config"
Environment Variables
Add these variables to your .env
file:
DATABASE_SYNC_REMOTE_USER_AND_HOST="forge@1.1.1.1" DATABASE_SYNC_REMOTE_DATABASE_USERNAME=forge DATABASE_SYNC_REMOTE_DATABASE_PASSWORD= DATABASE_SYNC_TEMPORARY_FILE_LOCATION_REMOTE=~/new_data.sql DATABASE_SYNC_TEMPORARY_FILE_LOCATION_LOCAL=~/Downloads/new_data.sql
Important: When connecting to a Forge-provisioned database server, you must use the main database user that was created during the initial server provisioning. Other database users created afterward may not have the necessary privileges to execute the required database commands for synchronization.
Usage
Basic Synchronization
To sync your remote database to local:
php artisan db-sync
By default, the package uses batch file transfers for optimal performance, transferring all table data in a single file to minimize network overhead.
Advanced Options
The sync command supports several options:
php artisan db-sync [options]
Available options:
--date
: Sync records from a specific date--suite
: Use a predefined suite of tables--table
: Sync a specific table--tenant
: Specify tenant for multi-tenant applications--skip-landlord
: Skip landlord database in multi-tenant setup--full-sync
: Sync the full table without a date constraint--status
: View the sync history and status for all tables--individual-transfers
: Use individual file transfers for each table (legacy behavior)
Per-Table Sync Tracking
The package now tracks the last sync date for each individual table, preventing data loss when syncing single tables. This means:
- Each table maintains its own sync history
- When syncing a specific table with
--table
, only that table's sync date is considered - The package automatically falls back to the global sync date for backward compatibility
- You can view the sync status of all tables using the
--status
option - Sync timestamps are captured at the start of the process to prevent missing data created during sync
Viewing Sync Status
To see the last sync date for all tables:
php artisan db-sync --status
This will display a table showing each table name and its last sync date, helping you track which tables have been synced recently and which might need attention.
How It Works
- Sync Start Timestamp: The sync timestamp is captured when the sync process begins
- Individual Table Tracking: Each table's sync date is stored separately in the cache file
- Atomic Updates: Cache is only updated when the entire sync process completes successfully
- Automatic Fallback: If no table-specific date exists, the global sync date is used
- Backward Compatibility: Existing installations continue to work without any changes
- Debug Information: When running with
-vvv
(debug mode), you'll see which sync date is being used for each table - Error Recovery: If sync fails, the cache remains unchanged with the previous sync dates
File Transfer Optimization
The package uses batch file transfers by default to minimize network overhead:
- Batch Mode (Default): All tables are dumped to a single file and transferred once
- Individual Mode: Each table is transferred separately (legacy behavior)
Configuration
Control the transfer mode in config/database-sync.php
:
'file_transfer_mode' => 'batch', // or 'individual'
Or via environment variable:
DATABASE_SYNC_FILE_TRANSFER_MODE=batch
Benefits of Batch Transfer
- Reduced network overhead: Single file transfer instead of multiple
- Faster sync times: Especially noticeable with many tables
- Better compression: SSH compression works more efficiently on larger files
- Lower resource usage: Fewer process spawns and file operations
When Individual Transfers Are Used
- Single table sync (
--table=tablename
) - Explicit override (
--individual-transfers
) - Config set to
individual
mode
Table Configuration
You can exclude specific tables from synchronization in the config/database-sync.php
file:
'tables' => [ 'ignore' => [ 'action_events', 'jobs', 'telescope_entries', 'password_resets', ], ],
Synchronization Suites
Define custom synchronization suites in the configuration file to group tables for specific sync tasks:
'suites' => [ 'orders' => [ 'orders', 'order_items', ], ],
Then use the suite option:
php artisan db-sync --suite=orders
Multi-Tenant Support
The package supports multi-tenant architectures. Enable it in the configuration:
'multi_tenant' => [ 'landlord' => [ 'database_name' => 'marshmallow_landord', 'tables' => [ 'ignore' => [ 'action_events', ], ], ], 'tenants' => [ 'database_names' => [ 'marshmallow_nl' => [ 'tables' => [ 'ignore' => [ 'users', ], ], ], 'marshmallow_dev', 'marshmallow_io', ], 'tables' => [ 'ignore' => [ 'logs', ], ], ], ],
Configure tenant-specific settings in your configuration file and use the --tenant
option to sync specific tenant databases:
php artisan db-sync --tenant="marshmallow_nl" --skip-landlord php artisan db-sync --tenant="marshmallow_nl" --skip-landlord --suite=orders
Testing
This package uses Pest PHP for testing. To run the tests:
composer test
To run tests with coverage report:
composer test-coverage
Test Structure
The test suite includes:
-
Unit Tests: Testing individual components
Config
class testsDatabaseSync
class tests- Other utility classes
-
Feature Tests: Testing the package functionality
- Command execution tests
- Multi-tenant functionality
- Suite configurations
- Table filtering
Writing Tests
To add new tests, create a new test file in either the tests/Unit
or tests/Feature
directory. The package uses Pest's expressive syntax:
test('your test description', function () { // Your test code expect($result)->toBe($expected); });
Security
- Never commit sensitive database credentials to version control
- Always use environment variables for sensitive information
- Ensure proper access controls on both remote and local databases
Support
For support, please email stef@marshmallow.dev
License
The MIT License (MIT). Please see License File for more information.