Jonathan Scheiber

Upgrade Doctrine Migrations from 2.x to 3.0

Read time: 6 minutes 14 comments
Upgrade Doctrine Migrations from 2.x to 3.0

Following the major upgrade of Doctrine Migrations and its Symfony bundle from version 2.x to 3.0, several issues have arisen. Some easy to adjust, others were more difficult. Let's see how to fix them all.

First steps after upgrade

After upgrading Doctrine Migrations and its Symfony bundle, you will probably encounter the following error message (if you didn't change the config in config/packages/doctrine_migrations.yaml):

Unrecognized options "dir_name, namespace, table_name" under "doctrine_migrations". Available options are "all_or_nothing", "check_database_platform", "connection", "custom_template", "em", "factories", "migrations", "migrations_paths", "organize_migrations", "services", "storage".

To fix this, check the upgrade guide available in bundle repository.

Sync migrations table structure

Then, when you will eg. check diff (with bin/console doctrine:migrations:diff), you will probably see this error:

The metadata storage is not up to date, please run the sync-metadata-storage command to fix this issue.

Table structure changed while 3.0 upgrade: now, instead of storing only the migration's version number, it will store the class' FQCN (namespace + class name) of executed migrations. Meaning that instead of seeing 20200603064733 in the migrations table, you will see DoctrineMigrations\Version20200603064733.

Hopefully the error message is pretty clear: you have to run the bin/console doctrine:migrations:sync-metadata-storage command.

However if the command does not work, you probably misconfigured (or didn't configure at all) DBMS version in Doctrine configuration (server_version parameter). In my case I have to specify mariadb-10.3.22. After setting this the command works well.

Other problems encountered

If you didn't set a specific table name in migrations config, in certain circumstances my table was named migration_versions, but in the default config it apparently changed to doctrine_migration_versions at some point. So if your current table is named migration_versions and nothing is configured in your config file, don't forget to configure it.

If you get the following error message:

An exception occurred while executing 'ALTER TABLE migration_versions ADD execution_time INT DEFAULT NULL, CHANGE version version VARCHAR(1024) NOT NULL, CHANGE executed_at executed_at DATETIME DEFAULT NULL':  
  SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 3072 bytes

You have to change the value of version_column_length in configuration to 191 (here is the related pull request).

Another important detail: the migrate command now displays an error message by default if no migrations are available (and therefore returns a code that is not 0, that can eg. stop your deploy). Solution: use the allow-no-migration option so the command displays a warning instead (and returns a 0 code): bin/console doctrine:migrations:migrate --allow-no-migration

Important update: it was fixed with 3.0.1 release :)

Handling several databases

In a big project we manage 9 different databases. Problem: options em and connection are not available anymore. These options used to define until version 2.x which connection/Entity Manager the migration commands would use.

After a few searches I realize the easiest way to handle this will be using factories.

As I already created proxy commands in the mentioned project, I only had to adapt them a little in order to handle this easily.

Please note I have for each Entity Manager a migration config file, stored in config/doctrine_migrations/. Eg. for the gps EM (that will be located at config/doctrine_migrations/gps.yaml) :

table_storage:
    table_name: _migration_versions
migrations_paths:
    'Application\Migrations\Gps': 'database/migrations/gps'
organize_migrations: year

Here is a proxy example for the migrate command:

<?php

declare(strict_types=1);

namespace App\Command\Migrations;

use Doctrine\Common\Persistence\ManagerRegistry;
use Doctrine\Migrations\Configuration\EntityManager\ExistingEntityManager;
use Doctrine\Migrations\Configuration\Migration\YamlFile;
use Doctrine\Migrations\DependencyFactory;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\ArrayInput;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;

final class MigrateCommand extends Command
{
    private ManagerRegistry $registry;

    public function __construct(ManagerRegistry $registry)
    {
        parent::__construct();
        $this->registry = $registry;
    }

    protected function configure(): void
    {
        $this
            ->setName('app:migrations:migrate')
            ->setDescription('Proxy to launch doctrine:migrations:migrate command as it would require a "configuration" option, and we can\'t define em/connection in config.')
            ->addArgument('em', InputArgument::REQUIRED, 'Name of the Entity Manager to handle.')
            ->addArgument('version', InputArgument::OPTIONAL, 'The version number (YYYYMMDDHHMMSS) or alias (first, prev, next, latest) to migrate to.', 'latest')
            ->addOption('dry-run', null, InputOption::VALUE_NONE, 'Execute the migration as a dry run.')
            ->addOption('query-time', null, InputOption::VALUE_NONE, 'Time all the queries individually.')
            ->addOption('allow-no-migration', null, InputOption::VALUE_NONE, 'Don\'t throw an exception if no migration is available (CI).')
        ;
    }

    protected function execute(InputInterface $input, OutputInterface $output): int
    {
        $newInput = new ArrayInput([
            'version'               => $input->getArgument('version'),
            '--dry-run'             => $input->getOption('dry-run'),
            '--query-time'          => $input->getOption('query-time'),
            '--allow-no-migration'  => $input->getOption('allow-no-migration'),
        ]);
        $newInput->setInteractive($input->isInteractive());
        $otherCommand = new \Doctrine\Migrations\Tools\Console\Command\MigrateCommand($this->getDependencyFactory($input));
        $otherCommand->run($newInput, $output);

        return 0;
    }

    private function getDependencyFactory(InputInterface $input): DependencyFactory
    {
        $em = $this->registry->getManager($input->getArgument('em'));
        $config = new YamlFile(__DIR__ . '/../../../config/doctrine_migrations/' . $input->getArgument('em') . '.yaml');

        return DependencyFactory::fromEntityManager($config, new ExistingEntityManager($em));
    }
}

First I define in command configuration an important argument: the Entity Manager name. Then I redefine the arguments and options of the base command I'm interested in.

In commands execution I define the arguments/options to pass, then create a new instance of the original command and pass it the correct Entity Manager. Finally the command is run.

Starting from this example, to launch the migrations of the gps EM, launch the following command: bin/console app:migrations:migrate gps --allow-no-migration

Et voilà! :) If you have any questions, comments, suggestions... Feel free to post a comment below :)

Comments

Posted by Rimi on 23/06/2020 at 10:27.
Thank you so much, very helpful article.
The still remaining one for me is factories. I am still new in symfony, I will give it a try and will let you know.

Thank you again!
Posted by Jenni on 23/06/2020 at 15:51.
Merci !

J'ai eu cette erreur 'Unrecognized options', grâce à ton article en 2 sec, j'ai résolu le problème de mise à jour de Doctrine.
Posted by Wolf on 22/07/2020 at 14:29.
Merci,
Sujet très bien abordé et mon problème a été résolu !
Posted by Fabrizio on 31/08/2020 at 17:34.
Hi Jonathan,
with Symfony 5.1.3 and two connection and two entity manager (example: default and booking_manager), how do i launch the command
php bin/console make:entity
using the entity manager booking_manager?

Best regards
Posted by jmsche on 06/09/2020 at 10:26.
Hi Fabrizio, the make:entity command does not need to know about your entity managers as Doctrine identifies the managers according to the namespace of your entities.
Posted by cBlondel on 24/11/2020 at 02:05.
Bonjour Jonathan,
Merci pour cette article !
Est-ce que vous utilisez la commande bin/console doctrine:migrations:diff avec des options spécifiques (--configuration, --db-configuration...) ou par l'intermédiaire d'une autre Commande Symfony ? De mon côté, cette commande ne génère pas mes migrations par entity manager. Elle regroupe le différentiel de l'ensemble de mes entités, sans tenir compte de leur appartenance à un entity manager.
Auriez-vous une idée ou des articles à me conseiller pour m'aider à avancer sur ce sujet ?
Merci pour votre aide.
Posted by jmsche on 24/11/2020 at 02:25.
Bonjour cBlondel,
Comme expliqué dans l'article, les commandes proxy créées manuellement vont charger un fichier de configuration spécifique ainsi que l'EntityManager lié, ce qui va normalement générer le diff pour les entités de l'EntityManager ciblé.
Le problème actuel de la v3.0 par rapport à la v2.x c'est qu'on ne peut pas cibler l'EntityManager ou la Connection qu'on veut via une option de la commande. Une PR est en cours sur le bundle doctrine/doctrine-migrations-bundle mais malheureusement pas encore fusionnée.
Posted by cBlondel on 24/11/2020 at 09:11.
Bonjour jmsche,
Merci pour votre réponse. En effet, j'ai vu que la version 3.1 de DoctrineMigrationsBundle permettra de renseigner de nouveau un EntityManager.
Pour résumer, dès que l'on souhaite communiquer l'EntityManager à une commande doctrine, on créé un proxy de cette commande (un proxy pour la commande doctrine:migrations:migrate, et un autre pour la commande doctrine:migrations:diff) ?
Merci
Posted by jmsche on 24/11/2020 at 09:29.
C'est bien cela :)
La commande proxy permet également de spécifier un fichier de configuration, qui permet notamment de spécifier le chemin de stockage des migrations en fonction de l'EntityManager, ce qui est plus propre à mon goût :)
Posted by Ludi on 20/12/2020 at 11:17.
Thanks a lot for your post.
Can you please provide the other commands as well?
e.g. app:migrations:diff
Posted by jmsche on 20/12/2020 at 13:40.
Hi Ludi,
The commands are pretty straight forward to write.
I wrote this gist a while ago, maybe this will suit your needs? https://gist.github.com/jmsche/2baf98b35eac936b0f6e77378c56f934
Posted by MushuLeDragon on 17/03/2021 at 12:07.
Bonjour,
Merci pour ce partage, ca correspond exactement à ma problématique.

Mais je voudrai savoir comment on peut faire pour la génération de fichiers de migration (`php bin/console make:migration`), sachant que actuellement je dois changer manuellement le fichier `doctrine_migrations.yaml` pour que la base de données corresponde, sachant que pour les migrations via doctrine je n'ai plus à le faire manuellement.

project
| - migration
| | - db1
| | - Version999998.php
| | - Version999999.php
| | - db2
| | - Version666666.php
| - public
| - src

Le but serait de n'avoir plus qu'une commande custom pour générer mes migrations et juste spécifier en argument la base de données associée (et donc les bonnes entités).

Merci
Posted by jmsche on 17/03/2021 at 13:26.
Bonjour MushuLeDragon,

Pour la génération de migrations, il suffit de créer un proxy pour la commande diff présente dans Doctrine Migrations.
Posted by MushuLeDragon on 17/03/2021 at 13:48.
Ohlalalala !
Je ne savais pas que d:m:diff faisait la même chose que make:migration !!

Merci infiniement

Add a comment