Bug #63669
Bug #63692: Memory consumption while bulk inserting
High memory consumption in DatabaseConnection->getDateTimeFormats while bulk inserting
| Status: | Resolved | Start date: | 2014-12-08 | |
|---|---|---|---|---|
| Priority: | Must have | Due date: | ||
| Assigned To: | - | % Done: | 100% |
|
| Category: | Performance | Spent time: | - | |
| Target version: | next-patchlevel | |||
| TYPO3 Version: | 6.2 | Is Regression: | No | |
| PHP Version: | Sprint Focus: | |||
| Complexity: | easy |
Description
Multiple calls from
- sysext/backend/Classes/Form/DataPreprocessor.php
- sysext/backend/Classes/Utility/BackendUtility.php
- sysext/core/Classes/DataHandling/DataHandler.php
do
$dateTimeFormats = $GLOBALS['TYPO3_DB']->getDateTimeFormats($table);
Currently there is no table handling implemented, but for EACH CALL of every field of every record to insert a new array is generated:
public function getDateTimeFormats($table) {
return array(
'date' => array(
'empty' => '0000-00-00',
'format' => 'Y-m-d'
),
'datetime' => array(
'empty' => '0000-00-00 00:00:00',
'format' => 'Y-m-d H:i:s'
)
);
}
Instead of creating this array multiple times it should be created as a property of the class:
/**
* the date and time formats compatible with the database in general
*
* @var array
*/
protected $dateTimeFormats = array(
'date' => array(
'empty' => '0000-00-00',
'format' => 'Y-m-d'
),
'datetime' => array(
'empty' => '0000-00-00 00:00:00',
'format' => 'Y-m-d H:i:s'
)
);
public function getDateTimeFormats($table) {
return $this->dateTimeFormats;
}
For bulk insert of 6500 records:
- memory usage of getDateTimeFormats() down from 79MB to 2kB
Associated revisions
[BUGFIX] Better performance of DatabaseConnection->getDateTimeFormats
Currently each call to DatabaseConnection->getDateTimeFormats creates a
new array with default dateTimeFormats. Put this information into a
static class member and return this instead.
Resolves: #63669
Releases: master, 6.2
Change-Id: I457ad6b68fb6919782f349550e9efd07143d0de7
Reviewed-on: http://review.typo3.org/35197
Reviewed-by: Alexander Opitz <opitz.alexander@googlemail.com>
Tested-by: Alexander Opitz <opitz.alexander@googlemail.com>
Reviewed-by: Stephan Großberndt <stephan@grossberndt.de>
Reviewed-by: Mathias Schreiber <mathias.schreiber@wmdb.de>
Tested-by: Mathias Schreiber <mathias.schreiber@wmdb.de>
Reviewed-by: Andreas Fernandez <a.fernandez@scripting-base.de>
Reviewed-by: Anja Leichsenring <aleichsenring@ab-softlab.de>
Tested-by: Anja Leichsenring <aleichsenring@ab-softlab.de>
[BUGFIX] Better performance of DatabaseConnection->getDateTimeFormats
Currently each call to DatabaseConnection->getDateTimeFormats creates a
new array with default dateTimeFormats. Put this information into a
static class member and return this instead.
Resolves: #63669
Releases: master, 6.2
Change-Id: I457ad6b68fb6919782f349550e9efd07143d0de7
Reviewed-on: http://review.typo3.org/35427
Reviewed-by: Anja Leichsenring <aleichsenring@ab-softlab.de>
Tested-by: Anja Leichsenring <aleichsenring@ab-softlab.de>
History
#1 Updated by Philipp Gampe 8 months ago
- Parent task set to #63692
#2 Updated by Gerrit Code Review 8 months ago
- Status changed from New to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35197
#3 Updated by Gerrit Code Review 8 months ago
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35197
#4 Updated by Gerrit Code Review 8 months ago
Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35427
#5 Updated by Stephan Großberndt 8 months ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset c90a324afb7f4a43b601af24bfeaa1aca3e0a628.