Bug #60494
editing gridelement in foreign language exhausts memory on large sites
Status: | Resolved | Start date: | 2014-07-23 | |
---|---|---|---|---|
Priority: | Should have | Due date: | ||
Assigned To: | - | % Done: | 100% |
|
Category: | - | Spent time: | - | |
Target version: | - | |||
TYPO3 Version: | 6.2 | Is Regression: | ||
PHP Version: | 5.3 | Sprint Focus: | ||
Complexity: |
Description
I have a site with very many pages. tt_content has about 14000 rows. When editing a gridelement in a foreign language typo3 takes too much memory. I had to set memory_limit to 512M.
With some debugging I found out that the problem is inside TYPO3\CMS\Backend\Utility\BackendUtility::getProcessedValue when it is called for $col = 'tx_gridelements_children'. The array $rParts gets very large because it is filled with all uids of tt_content where tx_gridelements_children is 0. For my site that means about 10000 elements. In the later processing of the uids php runs out of memory.
As a workaground I used the hook inside BackendUtility::getProcessedValue by setting
1$GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_befunc.php']['preProcessValue'][] = '&\WV\ImgReloaded\Hooks\GridElementsFix->preProcessValue';
inside ext_tables.php (inside an extension of mine)
The hook sets the field type to flex that no processing by BackendUtility::getProcessedValue will be done:
1<?php
2
3namespace WV\ImgReloaded\Hooks;
4
5// Hook done in ext_tables.php
6
7class GridElementsFix {
8
9 public function preProcessValue(&$params, &$ref) {
10 if (is_array($params) && array_key_exists('foreign_field', $params) && $params['foreign_field'] == 'tx_gridelements_container') {
11 $params['type'] = 'flex';
12 }
13 }
14
15}
I know the workaround is ugly. I hope someone can find a better fix.
(I had gridelements rev 1824f01414736595970f12f239fa76f4850288b3)
Related issues
Associated revisions
[BUGFIX] Fetch related records for existing uids only
Before fetching related records from a foreign table, we must ensure
that there is a uid to be considered, since otherwise we will fetch
any record having a 0 value for that field.
There are method calls in the core, that don't provide a uid, so the
default value 0 is used, i.e. when fetching children of translated
gridelements, causing memory outage when tt_content is > 10000 records.
Resolves: #60494
Releases: 6.3, 6.2
Change-Id: I1105964f98f79074bb37dc1921180b493fac4bc6
Reviewed-on: http://review.typo3.org/31781
Reviewed-by: Sascha Egerer <sascha@sascha-egerer.de>
Tested-by: Sascha Egerer <sascha@sascha-egerer.de>
Reviewed-by: Markus Klein <klein.t3@reelworx.at>
Tested-by: Markus Klein <klein.t3@reelworx.at>
[BUGFIX] Fetch related records for existing uids only
Before fetching related records from a foreign table, we must ensure
that there is a uid to be considered, since otherwise we will fetch
any record having a 0 value for that field.
There are method calls in the core, that don't provide a uid, so the
default value 0 is used, i.e. when fetching children of translated
gridelements, causing memory outage when tt_content is > 10000 records.
Resolves: #60494
Releases: 6.3, 6.2
Change-Id: I1105964f98f79074bb37dc1921180b493fac4bc6
Reviewed-on: http://review.typo3.org/31947
Reviewed-by: Markus Klein <klein.t3@reelworx.at>
Tested-by: Markus Klein <klein.t3@reelworx.at>
History
#1 Updated by Jo Hasenau about 1 year ago
- Status changed from New to Needs Feedback
IMHO this is a core bug, since the original type is "inline" and I don't see a good reason, why getProcessedValue should consider each element having a value of 0 here.
#2 Updated by Jo Hasenau about 1 year ago
The problem seems to be that $uid is 0 in this case and therefor
$records = self::getRecordsByField($theColConf['foreign_table'], $theColConf['foreign_field'], $uid);
fetches each record that is NOT child of a container.
So the question is, why is getProcessedValue called with a 0 value here.
#3 Updated by Jo Hasenau about 1 year ago
This is the definition of the method
/** * Returns a human readable output of a value from a record * For instance a database record relation would be looked up to display the title-value of that record. A checkbox with a "1" value would be "Yes", etc. * $table/$col is tablename and fieldname * REMEMBER to pass the output through htmlspecialchars() if you output it to the browser! (To protect it from XSS attacks and be XHTML compliant) * * @param string $table Table name, present in TCA * @param string $col Field name, present in TCA * @param string $value The value of that field from a selected record * @param integer $fixed_lgd_chars The max amount of characters the value may occupy * @param boolean $defaultPassthrough Flag means that values for columns that has no conversion will just be pass through directly (otherwise cropped to 200 chars or returned as "N/A") * @param boolean $noRecordLookup If set, no records will be looked up, UIDs are just shown. * @param integer $uid Uid of the current record * @param boolean $forceResult If BackendUtility::getRecordTitle is used to process the value, this parameter is forwarded. * @return string */ static public function getProcessedValue($table, $col, $value, $fixed_lgd_chars = 0, $defaultPassthrough = 0, $noRecordLookup = FALSE, $uid = 0, $forceResult = TRUE) {
And these are different use cases and how it is called:
$defaultLanguageValue = BackendUtility::getProcessedValue($table, $field, $this->defaultLanguageData[$table . ':' . $row['uid']][$field], 0, 1); $defaultLanguageValue = BackendUtility::getProcessedValue($table, $field, $this->additionalPreviewLanguageData[$table . ':' . $row['uid']][$previewLanguage['uid']][$field], 0, 1); $diffres = $t3lib_diff_Obj->makeDiffDisplay( BackendUtility::getProcessedValue($table, $field, $dLVal['old'][$field], 0, 1), BackendUtility::getProcessedValue($table, $field, $dLVal['new'][$field], 0, 1) );
As you can see they are using just 5 out of the 8 possible parameters, so $uid will always be 0, which is just plain wrong for the case of inline elements pointing to a foreign table.
I will move this to the core tracker.
#4 Updated by Jo Hasenau about 1 year ago
- Project changed from Grid Elements to Core
#5 Updated by Jo Hasenau about 1 year ago
This is not just about Gridelements, but AFAICS about any element using fields of type inline.
#6 Updated by Jo Hasenau about 1 year ago
IMHO it should be enough to introduce a check for $uid being available in line 2080 of typo3/sysext/backend/Classes/Utility/BackendUtility.php, since we need to check that for existing uids only.
#7 Updated by Gerrit Code Review about 1 year ago
- Status changed from Needs Feedback to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/31781
#8 Updated by Gerrit Code Review about 1 year ago
Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/31947
#9 Updated by Jo Hasenau about 1 year ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset 169f8b912751998b72236be85f1ff77e092ae8a2.