Bug #63745

Bug #63692: Memory consumption while bulk inserting

date/datetime checks in DataHandler->checkValue_input lead to high memory use while bulk inserting

Added by Stephan Großberndt 8 months ago. Updated 8 months ago.

Status:Resolved Start date:2014-12-10
Priority:Should have Due date:
Assigned To:- % Done:

100%

Category:Performance Spent time: -
Target version:next-patchlevel
TYPO3 Version:6.2 Is Regression:No
PHP Version: Sprint Focus:
Complexity:easy

Description

$dateTimeFormats are fetched for every input field (altough not needed) and the expensive GeneralUtility::inList-check for date,datetime-fields is executed twice:

public function checkValueForInput($res, $value, $tcaFieldConf, $table, $id, $realPid, $field = '') {
    // Handle native date/time fields
    $dateTimeFormats = $GLOBALS['TYPO3_DB']->getDateTimeFormats($table);
    if (isset($tcaFieldConf['dbType']) && GeneralUtility::inList('date,datetime', $tcaFieldConf['dbType'])) {
        // Convert the date/time into a timestamp for the sake of the checks
        $emptyValue = $dateTimeFormats[$tcaFieldConf['dbType']]['empty'];
        $format = $dateTimeFormats[$tcaFieldConf['dbType']]['format'];
        // At this point in the processing, the timestamps are still based on UTC
        $timeZone = new \DateTimeZone('UTC');
        $dateTime = \DateTime::createFromFormat('!' . $format, $value, $timeZone);
        $value = $value === $emptyValue ? 0 : $dateTime->getTimestamp();
    }

    [...]

    // Handle native date/time fields
    if (isset($tcaFieldConf['dbType']) && GeneralUtility::inList('date,datetime', $tcaFieldConf['dbType'])) {
        // Convert the timestamp back to a date/time
        $emptyValue = $dateTimeFormats[$tcaFieldConf['dbType']]['empty'];
        $format = $dateTimeFormats[$tcaFieldConf['dbType']]['format'];
        $res['value'] = $res['value'] ? date($format, $res['value']) : $emptyValue;
    }

Associated revisions

Revision 915427f1
Added by Stephan Großberndt 8 months ago

[BUGFIX] Do not execute getDateTimeFormats for each input

Fetches the dateTimeFormats only for date- and datetime-fields instead
of unconditionally for each input.

Resolves: #63745
Releases: master, 6.2
Change-Id: I18fcdb8a991ebb82891d9da06a4ef2b01fbc104f
Reviewed-on: http://review.typo3.org/35253
Reviewed-by: Markus Klein <>
Tested-by: Markus Klein <>
Reviewed-by: Wouter Wolters <>
Tested-by: Wouter Wolters <>

Revision 08c9282c
Added by Stephan Großberndt 8 months ago

[BUGFIX] Do not execute getDateTimeFormats for each input

Fetches the dateTimeFormats only for date- and datetime-fields instead
of unconditionally for each input.

Resolves: #63745
Releases: master, 6.2
Change-Id: I18fcdb8a991ebb82891d9da06a4ef2b01fbc104f
Reviewed-on: http://review.typo3.org/35320
Reviewed-by: Markus Klein <>
Tested-by: Markus Klein <>

History

#1 Updated by Wouter Wolters 8 months ago

  • Parent task set to #63692

#2 Updated by Gerrit Code Review 8 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#3 Updated by Gerrit Code Review 8 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#4 Updated by Gerrit Code Review 8 months ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#5 Updated by Gerrit Code Review 8 months ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#6 Updated by Gerrit Code Review 8 months ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#7 Updated by Gerrit Code Review 8 months ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35253

#8 Updated by Gerrit Code Review 8 months ago

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35320

#9 Updated by Stephan Großberndt 8 months ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#10 Updated by Gerrit Code Review 8 months ago

  • Status changed from Resolved to Under Review

Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35320

#11 Updated by Gerrit Code Review 8 months ago

Patch set 3 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35320

#12 Updated by Stephan Großberndt 8 months ago

  • Status changed from Under Review to Resolved

Also available in: Atom PDF