Disabling the Doctrine SQL logger
These days I'm writing a data import job for a Symfony application that uses Doctrine both for database abstraction and for object mapping.
The task handles a fairly large amount of data that can be up to multiple GB and is supposed to be invoked through a Symfony command which means it is run from the command line.
What first looked to be quite easy - read data from files, create objects from it and let Doctrine do the database persisting - caused me two days of headache due to the fact that the small script exhausted the set memory limit within minutes. It just made no sense that the memory for data worth of 1MB would actually have a foot print of 10 MB inside my application. Of course I suspected a memory leak even though I did my best to make it as easy as possible for the garbage collector to free my objects (e.g. regularly calling clear() on my entity manager).
After many hours of research I stumbled upon a blog post by Konrad Podgórski that provided the needed hint:
Doctrine writes a log that keeps track of every SQL string it has sent to the database and keeps this log in system memory all the time by default. Although this might indeed be very handy, for me it was the reason for multiple 100 MB of wasted memory.
Next to the mentioned DebugStack logger Doctrine comes with an echo logger (Doctrine\DBAL\Logging\EchoSQLLogger) and you even could implement one yourself. But as I don't need its functionality it was reasonable to just disable it:
$entityManager = $this->getContainer()->get('doctrine')->getManager(); $entityManager->getConnection()->getConfiguration()->setSQLLogger(null);
That completely solved my problem.