Bug #63669

Bug #63692: Memory consumption while bulk inserting

High memory consumption in DatabaseConnection->getDateTimeFormats while bulk inserting

Added by Stephan Großberndt 8 months ago. Updated 8 months ago.

Status:Resolved Start date:2014-12-08
Priority:Must have Due date:
Assigned To:- % Done:

100%

Category:Performance Spent time: -
Target version:next-patchlevel
TYPO3 Version:6.2 Is Regression:No
PHP Version: Sprint Focus:
Complexity:easy

Description

Multiple calls from

  • sysext/backend/Classes/Form/DataPreprocessor.php
  • sysext/backend/Classes/Utility/BackendUtility.php
  • sysext/core/Classes/DataHandling/DataHandler.php

do

$dateTimeFormats = $GLOBALS['TYPO3_DB']->getDateTimeFormats($table);

Currently there is no table handling implemented, but for EACH CALL of every field of every record to insert a new array is generated:

    public function getDateTimeFormats($table) {
        return array(
            'date' => array(
                'empty' => '0000-00-00',
                'format' => 'Y-m-d'
            ),
            'datetime' => array(
                'empty' => '0000-00-00 00:00:00',
                'format' => 'Y-m-d H:i:s'
            )
        );
    }

Instead of creating this array multiple times it should be created as a property of the class:

    /**
     * the date and time formats compatible with the database in general
     * 
     * @var array
     */
    protected $dateTimeFormats = array(
        'date' => array(
            'empty' => '0000-00-00',
            'format' => 'Y-m-d'
        ),
        'datetime' => array(
            'empty' => '0000-00-00 00:00:00',
            'format' => 'Y-m-d H:i:s'
        )
    );

    public function getDateTimeFormats($table) {
        return $this->dateTimeFormats;
    }

For bulk insert of 6500 records:

  • memory usage of getDateTimeFormats() down from 79MB to 2kB

Associated revisions

Revision c90a324a
Added by Stephan Großberndt 8 months ago

[BUGFIX] Better performance of DatabaseConnection->getDateTimeFormats

Currently each call to DatabaseConnection->getDateTimeFormats creates a
new array with default dateTimeFormats. Put this information into a
static class member and return this instead.

Resolves: #63669
Releases: master, 6.2
Change-Id: I457ad6b68fb6919782f349550e9efd07143d0de7
Reviewed-on: http://review.typo3.org/35197
Reviewed-by: Alexander Opitz <>
Tested-by: Alexander Opitz <>
Reviewed-by: Stephan Großberndt <>
Reviewed-by: Mathias Schreiber <>
Tested-by: Mathias Schreiber <>
Reviewed-by: Andreas Fernandez <>
Reviewed-by: Anja Leichsenring <>
Tested-by: Anja Leichsenring <>

Revision 2523f3c6
Added by Stephan Großberndt 8 months ago

[BUGFIX] Better performance of DatabaseConnection->getDateTimeFormats

Currently each call to DatabaseConnection->getDateTimeFormats creates a
new array with default dateTimeFormats. Put this information into a
static class member and return this instead.

Resolves: #63669
Releases: master, 6.2
Change-Id: I457ad6b68fb6919782f349550e9efd07143d0de7
Reviewed-on: http://review.typo3.org/35427
Reviewed-by: Anja Leichsenring <>
Tested-by: Anja Leichsenring <>

History

#1 Updated by Philipp Gampe 8 months ago

  • Parent task set to #63692

#2 Updated by Gerrit Code Review 8 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35197

#3 Updated by Gerrit Code Review 8 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35197

#4 Updated by Gerrit Code Review 8 months ago

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35427

#5 Updated by Stephan Großberndt 8 months ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF