Bug #43540

Task #52304: Performance issues

TS is fetched from cache incorrectly sometimes

Added by Dmitry Dulepov over 2 years ago. Updated over 1 year ago.

Status:Resolved Start date:2012-12-03
Priority:Must have Due date:
Assigned To:Dmitry Dulepov % Done:

100%
Category:TypoScript Spent time: -
Target version:- Estimated time:8.00 hours
TYPO3 Version:6.0 Is Regression:No
PHP Version: Sprint Focus:
Complexity:hard

Description

If $GLOBALS['TYPO3_CONF_VARS']['FE']['defaultTypoScript_constants'] or $GLOBALS['TYPO3_CONF_VARS']['FE']['defaultTypoScript_setup'] are set in the Install tool, cached TS will not be found in cache. This would result in TS parsing every time when there are USER_INT objects on the page. Generally it will slow down the page with USER_INT objects by at least 300% comparing to the non-cached version.

More information

When template parser processes the TS, it makes a hash to load a cached version of TS:

$this->runThroughTemplates($theRootLine);
...
$rowSumHash = md5('ROWSUM:' . serialize($this->rowSum));
$result = \TYPO3\CMS\Frontend\Page\PageRepository::getHash($rowSumHash);

There TS can be fetched from the cache. If nothing is returned, the TS is parsed and stored into the cache:

$this->generateConfig();
...
$rowSumHash = md5('ROWSUM:' . serialize($this->rowSum));
\TYPO3\CMS\Frontend\Page\PageRepository::storeHash($rowSumHash, serialize($cc['all']), 'TMPL_CONDITIONS_ALL');

Function generateConfig() prepends values from the above-mentioned variables before processing the data. The earlier called function runThroughtTemplates() does not do that. If the variables are empty, it does not make any difference and everything is cached. But if those variables are not empty (even if there is a single space), the $this->rowSum will be different in two quoted code extracts above. Thus TS will be stored with one hash value and the next request will try to retrieve it with another hash value. This means that cached TS will never be retrieved and will be always parsed.

We were able to decrease page generation from 2.88s to 0.8s by fixing the issue. 2 seconds make a huge difference in speed and load of the web server.

Related issues

blocks Core - Bug #53352: Template Analyzer produces wrong results Resolved 2013-11-05

Associated revisions

Revision 4fbb2b58
Added by Dmitry Dulepov over 2 years ago

[BUGFIX] TS is fetched from cache incorrectly sometimes

If $TYPO3_CONF_VARS['FE']['defaultTypoScript_constants'] or
$TYPO3_CONF_VARS['FE']['defaultTypoScript_setup'] are set in the Install
tool, cached TS will not be found in cache. This would result in TS
parsing every time when there are USER_INT objects on the page.
Generally it will slow down the page with USER_INT objects by at least
300% comparing to the cached version.

Change-Id: Ia0fd65a35897c71d60c48c0b03098ce67ad16c70
Resolves: #43540
Releases: 4.5, 4.6, 4.7, 6.0, 6.1
Reviewed-on: https://review.typo3.org/16939
Reviewed-by: Adrian Dymorz
Tested-by: Adrian Dymorz
Reviewed-by: Stefan Neufeind
Reviewed-by: Anja Leichsenring
Tested-by: Anja Leichsenring
Reviewed-by: Sebastian Michaelsen
Reviewed-by: Dmitry Dulepov
Tested-by: Dmitry Dulepov

Revision 09f3b6af
Added by Dmitry Dulepov almost 2 years ago

[BUGFIX] TS is fetched from cache incorrectly sometimes

If $TYPO3_CONF_VARS['FE']['defaultTypoScript_constants'] or
$TYPO3_CONF_VARS['FE']['defaultTypoScript_setup'] are set through
the PHP code, cached TS will not be found in cache. This would
result in TS parsing every time when there are USER_INT objects
on the page. It may slow down the page with USER_INT objects
by about 300% comparing to the cached version.

This change is re-added after it got reverted due to lack of reviews in
Ia0fd65a35897c71d60c48c0b03098ce67ad16c70.

Change-Id: Ief3a065451644423b236489729716b906f3d4500
Resolves: #43540
Releases: 4.5, 4.7, 6.0, 6.1, 6.2
Reviewed-on: https://review.typo3.org/18072
Reviewed-by: Jigal van Hemert
Tested-by: Jigal van Hemert

Revision 6b2512a9
Added by Dmitry Dulepov almost 2 years ago

[BUGFIX] TS is fetched from cache incorrectly sometimes

If $TYPO3_CONF_VARS['FE']['defaultTypoScript_constants'] or
$TYPO3_CONF_VARS['FE']['defaultTypoScript_setup'] are set through
the PHP code, cached TS will not be found in cache. This would
result in TS parsing every time when there are USER_INT objects
on the page. It may slow down the page with USER_INT objects
by about 300% comparing to the cached version.

This change is re-added after it got reverted due to lack of reviews in
Ia0fd65a35897c71d60c48c0b03098ce67ad16c70.

Change-Id: Ief3a065451644423b236489729716b906f3d4500
Resolves: #43540
Releases: 4.5, 4.7, 6.0, 6.1, 6.2
Reviewed-on: https://review.typo3.org/24177
Reviewed-by: Jigal van Hemert
Tested-by: Jigal van Hemert

Revision 8e1ea888
Added by Dmitry Dulepov almost 2 years ago

[BUGFIX] TS is fetched from cache incorrectly sometimes

If $TYPO3_CONF_VARS['FE']['defaultTypoScript_constants'] or
$TYPO3_CONF_VARS['FE']['defaultTypoScript_setup'] are set through
the PHP code, cached TS will not be found in cache. This would
result in TS parsing every time when there are USER_INT objects
on the page. It may slow down the page with USER_INT objects
by about 300% comparing to the cached version.

This change is re-added after it got reverted due to lack of reviews in
Ia0fd65a35897c71d60c48c0b03098ce67ad16c70.

Change-Id: Ief3a065451644423b236489729716b906f3d4500
Resolves: #43540
Releases: 4.5, 4.7, 6.0, 6.1, 6.2
Reviewed-on: https://review.typo3.org/24178
Reviewed-by: Jigal van Hemert
Tested-by: Jigal van Hemert

Revision f8f155e5
Added by Jigal van Hemert almost 2 years ago

[BUGFIX] TS is fetched from cache incorrectly sometimes

If $TYPO3_CONF_VARS['FE']['defaultTypoScript_constants'] or
$TYPO3_CONF_VARS['FE']['defaultTypoScript_setup'] are set through
the PHP code, cached TS will not be found in cache. This would
result in TS parsing every time when there are USER_INT objects
on the page. It may slow down the page with USER_INT objects
by about 300% comparing to the cached version.

This change is re-added after it got reverted due to lack of reviews in
Ia0fd65a35897c71d60c48c0b03098ce67ad16c70.

Change-Id: Ief3a065451644423b236489729716b906f3d4500
Resolves: #43540
Releases: 4.5, 4.7, 6.0, 6.1, 6.2
Reviewed-on: https://review.typo3.org/24182
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind

History

#1 Updated by Dmitry Dulepov over 2 years ago

Should be read: "...comparing to the cached version".

#2 Updated by Gerrit Code Review over 2 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/16939

#3 Updated by Gerrit Code Review over 2 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/16939

#4 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch TYPO3_6-0 has been pushed to the review server.
It is available at https://review.typo3.org/18056

#5 Updated by Dmitry Dulepov over 2 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 10 to 100

#6 Updated by Gerrit Code Review over 2 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at https://review.typo3.org/18059

#7 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at https://review.typo3.org/18061

#8 Updated by Gerrit Code Review over 2 years ago

Patch set 2 for branch TYPO3_4-7 has been pushed to the review server.
It is available at https://review.typo3.org/18059

#9 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch TYPO3_4-5 has been pushed to the review server.
It is available at https://review.typo3.org/18062

#10 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#11 Updated by Gerrit Code Review over 2 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#12 Updated by Gerrit Code Review over 2 years ago

Patch set 3 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#13 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch master_new has been pushed to the review server.
It is available at https://review.typo3.org/18621

#14 Updated by Gerrit Code Review over 2 years ago

Patch set 4 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#15 Updated by Gerrit Code Review about 2 years ago

Patch set 5 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#16 Updated by Markus Klein almost 2 years ago

  • Parent task set to #52338
  • Is Regression set to No

#17 Updated by Markus Klein almost 2 years ago

  • Parent task changed from #52338 to #52304

#18 Updated by Kasper Ligaard almost 2 years ago

This solves the problem described in this thread: http://forum.typo3.org/index.php/m/695487/#page_top

#19 Updated by Gerrit Code Review almost 2 years ago

Patch set 6 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/18072

#20 Updated by Gerrit Code Review almost 2 years ago

Patch set 1 for branch TYPO3_6-1 has been pushed to the review server.
It is available at https://review.typo3.org/24177

#21 Updated by Gerrit Code Review almost 2 years ago

Patch set 1 for branch TYPO3_6-0 has been pushed to the review server.
It is available at https://review.typo3.org/24178

#22 Updated by Dmitry Dulepov almost 2 years ago

  • Status changed from Under Review to Resolved

#23 Updated by Gerrit Code Review almost 2 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at https://review.typo3.org/24182

#24 Updated by Gerrit Code Review almost 2 years ago

Patch set 1 for branch TYPO3_4-5 has been pushed to the review server.
It is available at https://review.typo3.org/24981

#25 Updated by Jigal van Hemert almost 2 years ago

  • Status changed from Under Review to Resolved

#26 Updated by Simon Schaufelberger over 1 year ago

Hey, Why is this NOT included in 4.5??? so sad...

#27 Updated by Markus Klein over 1 year ago

@Simon: It will be finally included.

Also available in: Atom PDF