Task #54251

Epic #55070: Workpackages

Epic #55065: WP: Overall System Performance (Backend and Frontend)

Bug #52949: Speed decrease since 4.5

GeneralUtility::array_merge_recursive_overrule has bad performance

Added by Markus Klein over 1 year ago. Updated over 1 year ago.

Status:Resolved Start date:2013-12-06
Priority:Should have Due date:
Assigned To:Markus Klein % Done:

100%
Category:Performance Spent time: -
Target version:6.2.0
TYPO3 Version:6.2 Complexity:easy
PHP Version:5.5 Sprint Focus:

Description

According to my profiling activities, GeneralUtility::array_merge_recursive_overrule has really bad performance. (~ 111 ms own time for a BE call, that ranks number 3 in the list of the most expensive calls based on own time sorting.)
Some tests show that this is due to many array copy actions during the recursive calls (foreach constructs with $val being a copy of possibly large subarrays).

I propose to optimize this method by:
  1. making the first parameter by reference thus working on the original array
  2. move the function from GeneralUtility to ArrayUtility
  3. deprecate the function in GeneralUtility, but still work on a copy of the first array there to preserve backwards compatibility
  4. change all calls in core to use the new method

This brings the own time of the function down to 42 ms on my test system, getting the function out of the expensive calls in TYPO3 CMS.

Related issues

related to Core - Task #57499: Improve deprecation message of GeneralUtitlity::array_mer... Resolved 2014-04-01
related to Core - Bug #63812: Lower memory use in ArrayUtil->mergeRecursiveWithOverrule Resolved 2014-12-12

Associated revisions

Revision 2917b074
Added by Markus Klein over 1 year ago

[TASK] Improve performance of array_merge_recursive_overrule

The method GeneralUtility::array_merge_recursive_overrule()
always works on a copy of the given array(s). This is highly
inefficient when it comes to really big arrays and recursion.

This patches moves the functionality into the class ArrayUtility
and changes the behaviour to use a reference to the
original array.
All calls in the core are adjusted accordingly.

Furthermore we deprecate the method in GeneralUtility and
preserve backward compatibility.

Resolves: #54251
Releases: 6.2
Change-Id: I5499905593c2124897de5998be985e546a3d05ee
Reviewed-on: https://review.typo3.org/25986
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters
Reviewed-by: Ernesto Baschny
Tested-by: Ernesto Baschny

History

#1 Updated by Markus Klein over 1 year ago

  • Parent task set to #52949

#2 Updated by Gerrit Code Review over 1 year ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#3 Updated by Gerrit Code Review over 1 year ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#4 Updated by Gerrit Code Review over 1 year ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#5 Updated by Gerrit Code Review over 1 year ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#6 Updated by Gerrit Code Review over 1 year ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#7 Updated by Gerrit Code Review over 1 year ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#8 Updated by Gerrit Code Review over 1 year ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#9 Updated by Gerrit Code Review over 1 year ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#10 Updated by Gerrit Code Review over 1 year ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#11 Updated by Gerrit Code Review over 1 year ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#12 Updated by Markus Klein over 1 year ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF