Bug #63674

Bug #63692: Memory consumption while bulk inserting

High memory consumption in BackendUtility->explodeSoftRefParserList while bulk inserting

Added by Stephan Großberndt 8 months ago. Updated 7 months ago.

Status:Resolved Start date:2014-12-08
Priority:Should have Due date:
Assigned To:- % Done:

0%

Category:Performance Spent time: -
Target version:next-patchlevel
TYPO3 Version:6.2 Is Regression:No
PHP Version: Sprint Focus:
Complexity:no-brainer

Description

Due to improper checking for an empty value BackendUtility->explodeSoftRefParserList multiple trimExplodes per record are executed

static public function explodeSoftRefParserList($parserList) {
    // Looking for global parsers:
    if (is_array($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['softRefParser_GL'])) {
        $parserList = implode(',', array_keys($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['softRefParser_GL'])) . ',' . $parserList;
    }
    // Return immediately if list is blank:
    if (!strlen($parserList)) {
        return FALSE;
    }

Using only is_array when looking for global parsers and appending ',' . $parserList leads to $parserList = ',' which does not result in an early return but a full parse.

static public function explodeSoftRefParserList($parserList) {
    // Looking for global parsers:
    if (is_array($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['softRefParser_GL']) && count($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['softRefParser_GL']) > 0) {
        $parserList = implode(',', array_keys($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['GLOBAL']['softRefParser_GL'])) . ',' . $parserList;
    }
    // Return immediately if list is blank:
    if (!strlen($parserList)) {
        return FALSE;
    }

For bulk inserting of 6500 records:

  • 1 sec faster
  • memory_usage of explodeSoftRefParserList down by 4.5MB

Related issues

related to Core - Task #64109: Deprecate softRefParser_GL Resolved 2015-01-04

Associated revisions

Revision 522fba7d
Added by Stephan Großberndt 7 months ago

[BUGFIX] Speed up BackendUtility->explodeSoftRefParserList

Due to improper checking for an empty array in
BackendUtility->explodeSoftRefParserList multiple trimExplode-calls per
record are executed. Fixing this check leads to an early return and
speeds up bulk inserting.

Resolves: #63674
Releases: master, 6.2
Change-Id: Id8969c33d8ebb109018159ada6a277ab5bfdd13c
Reviewed-on: http://review.typo3.org/35187
Reviewed-by: Michael Oehlhof <>
Reviewed-by: Mathias Schreiber <>
Tested-by: Mathias Schreiber <>
Reviewed-by: Benjamin Mack <>
Tested-by: Benjamin Mack <>

Revision 3589da89
Added by Stephan Großberndt 7 months ago

[BUGFIX] Speed up BackendUtility->explodeSoftRefParserList

Due to improper checking for an empty array in
BackendUtility->explodeSoftRefParserList multiple trimExplode-calls per
record are executed. Fixing this check leads to an early return and
speeds up bulk inserting.

Resolves: #63674
Releases: master, 6.2
Change-Id: Id8969c33d8ebb109018159ada6a277ab5bfdd13c
Reviewed-on: http://review.typo3.org/35187
Reviewed-by: Michael Oehlhof <typo3@oehlhof.de>
Reviewed-by: Mathias Schreiber <mathias.schreiber@wmdb.de>
Tested-by: Mathias Schreiber <mathias.schreiber@wmdb.de>
Reviewed-by: Benjamin Mack <benni@typo3.org>
Tested-by: Benjamin Mack <benni@typo3.org>
(cherry picked from commit 522fba7db058c0c2783116d9772c2c2a561b6beb)
Reviewed-on: http://review.typo3.org/35775

History

#1 Updated by Philipp Gampe 8 months ago

  • Parent task set to #63692

#2 Updated by Gerrit Code Review 8 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35187

#3 Updated by Gerrit Code Review 8 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35187

#4 Updated by Gerrit Code Review 7 months ago

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35775

#5 Updated by Wouter Wolters 7 months ago

  • Status changed from Under Review to Resolved

Also available in: Atom PDF