Core

I can reproduce this in TYPO3 6.2 and with an own extension. Images are just not handled correctly, and in the translation of a record it should be possible to remove images or add new ones. But ONLY the images of the default record are shown in the frontend.

When using...

'l10n_mode' => 'noCopy'

...in TCA, the fields are actually not copied and not shown in the frontend when displaying the translated record, but if I manually add another image to the translated record, the default language images are shown again. So I guess this is, apart from other translation handling issues in extbase/TYPO3 a major bug.

I also tried:

'l10n_mode' => 'mergeIfNotBlank'

...but it doesn't work and only the default record images are shown.

A multilanguage CMS should really work with multiple languages IMHO.

Related to #59192

FAL image localization problem faced in the case of flexforms also. I've created a flexform element based on this http://wiki.typo3.org/File_Abstraction_Layer#FlexForm, but it has a problem when translating default content element records. When I translate default content element, the images are not copied to translated element. If we put "media" as the fieldname in flexform, it copied the images, but it shows some unwanted images along with that.

Workaround for the original problem:

Within \TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper::getConstraint() there is an equals-constraint with "$parentObject" - this is the object that has the FAL-relation as one of its properties. When the equals-constraint is evaluated, this constraint will be transformed into a SQL-where with the uid of "$parentObject" and that´s the reason why only the FAL-relations of the not-localized object will be used, the uid of the localized object will be ignored, but the FAL-relations of the localized objects are stored with the uid of the localized objects.

There are different ways to hotfix:
You can just extend DataMapper, inject your class with the "config.tx_extbase.objects"-TypoScript and override getConstraint() to use "$parentObject->_getProperty('_localizedUid')" in the equals-constraint.
For a less global change you could check the table and field names within getConstraint() and restrict this change to the fields where you need it.
A more object oriented approach would be to also extend AbstractDomainObject, write some kind of "constraint uid getter", integrate this method within DataMapper::getConstraint() and return the "_localizedUid" property in the model classes where required - you could also pass the fieldname as an argument.

as first solution, following Robert suggestions, i solved the problem modifyng \TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper::getConstraint()

line

$constraint = $query->equals($columnMap->getParentKeyFieldName(), $parentObject);

becomes

if ($columnMap->getColumnName() == 'fal_related_files') {
    $constraint = $query->equals($columnMap->getParentKeyFieldName(), $parentObject->_getProperty('_localizedUid'));
} else {
    $constraint = $query->equals($columnMap->getParentKeyFieldName(), $parentObject);
}

update: we can apply this temporary patch as a XCLASS (using it on 6.2.4)

in ext_localconf.php:

// XCLASS
// fix https://forge.typo3.org/issues/57272
$GLOBALS['TYPO3_CONF_VARS']['SYS']['Objects']['TYPO3\\CMS\\Extbase\\Persistence\\Generic\\Mapper\\DataMapper'] = array(
    'className' => 'Tx_Myext_XClass_DataMapper'
);

and then in /Classes/XClasses/DataMapper.php

<?php

/**
 * A mapper to map database tables configured in $TCA on domain objects.
 */
class Tx_Myext_XClass_DataMapper extends \TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper
{

    /**
     * Builds and returns the constraint for multi value properties.
     *
     * @param \TYPO3\CMS\Extbase\Persistence\QueryInterface $query
     * @param \TYPO3\CMS\Extbase\DomainObject\DomainObjectInterface $parentObject
     * @param string $propertyName
     * @param string $fieldValue
     * @param array $relationTableMatchFields
     * @return \TYPO3\CMS\Extbase\Persistence\Generic\Qom\ConstraintInterface $constraint
     */
    protected function getConstraint(\TYPO3\CMS\Extbase\Persistence\QueryInterface $query, \TYPO3\CMS\Extbase\DomainObject\DomainObjectInterface $parentObject, $propertyName, $fieldValue = '', $relationTableMatchFields = array())
    {
        $columnMap = $this->getDataMap(get_class($parentObject))->getColumnMap($propertyName);
        if ($columnMap->getParentKeyFieldName() !== NULL) {
           if ($columnMap->getColumnName() == 'fal_related_files') {
                $constraint = $query->equals($columnMap->getParentKeyFieldName(), $parentObject->_getProperty('_localizedUid'));
            } else {
                $constraint = $query->equals($columnMap->getParentKeyFieldName(), $parentObject);
            }

            if ($columnMap->getParentTableFieldName() !== NULL) {
                $constraint = $query->logicalAnd($constraint, $query->equals($columnMap->getParentTableFieldName(), $this->getDataMap(get_class($parentObject))->getTableName()));
            }
        } else {
            $constraint = $query->in('uid', \TYPO3\CMS\Core\Utility\GeneralUtility::intExplode(',', $fieldValue));
        }
        if (count($relationTableMatchFields) > 0) {
            foreach ($relationTableMatchFields as $relationTableMatchFieldName => $relationTableMatchFieldValue) {
                $constraint = $query->logicalAnd($constraint, $query->equals($relationTableMatchFieldName, $relationTableMatchFieldValue));
            }
        }
        return $constraint;
    }
}

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32718

Raphael Zschorsch wrote:

in the translation of a record it should be possible to remove images or add new ones. But ONLY the images of the default record are shown in the frontend.
...but it doesn't work and only the default record images are shown.

There are many things to consider with relations and translations. We can make most use cases work in Extbase (FAL images is the most obvious one), but should also make clear that the desired behavior heavily depends on the use case and extension developers may need to adapt annotations or TCA to make it work like desired (which in turn could make it hard for backports).

If you have a simple 1:n relation (a RelationalObject has exactly one AggregateRootObject):

AggregateRootObject (ARO) -> RelationalObject (RO)

Depending on your domain, you want different behaviors of the relations in case of localization

Case 1: Relations must not change. This means, that a specific ARO is always related to a specific RO in all localizations. This means that text fields are translatable (editable), but the relation is not editable for localizations to maintain consistency.

E.g.: Product -> Brand
It makes sense to translate the product and Brand description, but it does not make sense to localize the Product/Brand as for every localization the Product will always have the same Brand.

Case 2: Relations can change for localized entities. This means AROs can have relations to different ROs for every localization. This means not only text fields are localized but also relations (id fields or intermediate tables point to localized versions of an entity)
That is certainly more complicated, as additional behavior needs to be defined. Does not found relation on localized record mean the localization has none, or should a fallback be in place?
Accessing the ARO from a RO (inverse relation) should result in the localized ARO or the default? If it is the localized ARO, how about other relations that should behave like described in case 1?

Especially case 2 can be a nightmare to think through and define what is wanted, not to speak about implementing it.
"Unfortunately" images are now (with FAL) relations to entities and what we need is exactly case 2.
This is the issue we face now and we should solve it by well defining (and documenting) how things work (or should work) under which conditions.

Oh and btw. case 2 is not implemented in Extbase.

Here is what DataHandler currently does for relation fields, when "l10n_mode" is not set to "mergeIfNotBlank" or "exclude"

AggregateRootObject (default)   ->  RelationalObject (default)

AggregateRootObject (default)   ->  RelationalObject (default)
  |-> AggregateRootObject (L1)   ->  |-> RelationalObject (L1)

What the Extbase persistence does is fetching the records in the default language and performing a language overlay (it is a bit more complicated but it does not matter for now).

The example above would then work like a charm :

Fetch AggregateRootObject (default), overlay it, Fetch relations which is RelationalObject (default) and overlay it.

This means if your TCA is correct for your main object (News TCA for fal_media is at least inconsistent), then it works if you only need to localize relations, but not have completely different relations for localized entities.

When you need the latter, then you are in case 2 hell. E.g. if we would change it to behave like proposed and always fetch images for news records based on the localized record, then we would break existing installations that relied on the fact that translated news would not need images.

So the questions are:

1. What cases are there that need to be covered
1. How should the configuration for such cases be (re-use existing TCA properties or create new ones or use annotations)
1. How can we implement something in a backwards compatible way?

Hi, Helmut.

Thank you for a detailed description!

I'm still the one, who'd like to force "Case 2" to be handled by Extbase natively for any localization (including FAL) - not by hacks, workarounds or so.
And, sure, I see the complication of possible solution and I can imagine even more overhead in persistence.

But from the other hand this is not some rarely used feature, and it becomes hard for us to explain customer, why Extbase with FAL and 6.x better, then Pibase with usual file handling and 4.x, because customer misses features, that were existing for years in old TYPO3 versions. Sure, we can go back to 'file' and 'file_reference' and write own persistance layer for other localizable RO, but then I can ask myself a same question: "why da hell I need Extbase and FAL?".

So, answering your questions:

Does not found relation on localized record mean the localization has none, or should a fallback be in place?

I think, here we can re-define a TCA option 'l10n_mode' (or add Extbase annotation?), with such options: includeNewChildren - no relation means no fallback (so, only translated and new added relations will be shown), includeNewChildrenAndFallbacks - missing relation falls back to master record (so, translated, new added and non-translated relations will be shown.

Accessing the ARO from a RO (inverse relation) should result in the localized ARO or the default? If it is the localized ARO, how about other relations that should behave like described in case 1?

Here I may be wrong, but we should have the master ARO (uid) with translated fileds (title, description, etc.), like it is already done in Extbase. But, sure, a new option in TCA (or Extbase annotation) may be added for this.

What cases are there that need to be covered?

Being able to change relations in translated versions:
1. Exchange image in relation for translated version of website.
2. Add more new images in translated version of website.

How should the configuration for such cases be (re-use existing TCA properties or create new ones or use annotations)

Good question. And it opens even more complicated topic: why do we need a TCA at all? My dreams show fututre TYPO3 version (9.x, maybe) without TCA, but with only annotations in Extbase Domain Models. As result: we don't define validation rules, pre- and post- processings twice, but use single source for 3 ways: BE forms, FE forms and internal object creation via DataMapper. In such a case, I'd prefer annotations, but in real world we can stick to new TCA properties.

How can we implement something in a backwards compatible way?

In case of new TCA properties, this can be achieved by making default properties behave, like it works now: changing relations in child records will not affect output of these records.

Hi Viktor.

Viktor Livakivskyi wrote:

I'm still the one, who'd like to force "Case 2" to be handled by Extbase natively for any localization (including FAL) - not by hacks, workarounds or so.
And, sure, I see the complication of possible solution and I can imagine even more overhead in persistence.

We absolutely need a solution.

I think, here we can re-define a TCA option 'l10n_mode' (or add Extbase annotation?)

One or the other or something different ;)
The question is: What is a good option respecting backwards compatibility, that is easy to understand and predictable in the result.

Being able to change relations in translated versions:
1. Exchange image in relation for translated version of website.
2. Add more new images in translated version of website.

Sure this is one case. But what are the consequences especially for inverse relations (as described above). It could be the case that everything turns out to work without issues, but currently I can't tell.

Any help is appreciated, especially creating extensive functional test cases. Frans started working on that:

https://review.typo3.org/#/c/32718/

I updated the change yesterday providing a more generic solution, but the place where it should be configured is still not clear.

Functional tests should check the behavior with and without the (new?) configuration and also check related things like how it works with inverse relationships.

Would be great if you could help working on that.

Hi, Helmut.

Many thanks for your feedback.

Yes, I agree with you on having good overview over possible solutions and finding best and less migration-stressy one.
For the moment I see, that re-using of TCA option 'l10n_mode' may be handy, since everyone is familiar with it and nothing needs to be changed in existing installations, which rely on current behavior.

Inverse relations also will depend on same TCA option, so it might be a case, that entity1->entity2 relation is e.g. "includeNewChildren", but entity2->entity1 realtion will be "includeNewChildrenAndFallbacks", so we can fetch master entity1, if no translation is found.

I'd like to help with functional tests, but I'm not yet familiar with their concept - we've only started to invent unit-tests in our workflow. So, I'll come back later, when I have practice with functional tests and learn more the code, that needs to be tested.

I just applied this test to a 6.2 with EXT:news. My initial findings are reported in #59192. I'm working with sys_language_mode = strict.

This change basically enables the correct use of explicitely translated or added related items.

The only thing that is missing in my case is that the l10n_mode is taken into account. If the l10n_mode is set to "mergeIfNotBlank", I would expect TYPO3 to use the related items of the default language as long as no relation in the translated version is explicitely created. As soon as there is a relation in the translated version, only the item(s) in the translated version should be considered. This is how l10n_mode "mergeIfNotBlank" works IMO.

With that patch (d39c8b2d) in TYPO3 6.2.4 and 6.2.6 images in translated news are correctly shown. But this works only for news with an parent data in default language. With news without parent data in default language no image is shown. In list and single view.

I used the last news from github. My TypoScript configuration: sys_language_mode = strict, no sys_language_overlay.

The following behavior is also very special.

German is default language and English is the first additional language in my TYPO3-Site.

• I add a new news data with image in english. In frontend the news on the english page is visible but without image.
• I add a news data with image again but now in german language. I translate this data in english and add a image (image from default langauge was not copied). This news is showing in english frontend with image. Everything is fine.
• I delete the relation between these german and english news and have now one separate german news and one separate english news. The english news has no information about his parent localization now. But in english frontend the same news is showing with image. Everything is fine. I think its the same situation like with the first news? Seeming not.
• Now i delete the german news data which was the localized parent data before. Now in english frontend the complete news is not there? Nothing is shown.

I hope anybody can understand me and can explain this.

Ok. I understand now.
I've got checked sys_file_reference. Here are my results:

1. i create a new news record in default language, then save it.
2. now i add a image to the news, change the language (1=english) of the record and save it.
3. in sys_file_reference the field sys_language_uid is filled with "0" default language but the news record is now in english

conclusion: no image in frontend

1. i create the news record and change the language to english before i saving this
2. now i go and add an image and saving the news
3. the right sys_language_uid is filled in sys_file_reference.

conclusion: image is shown in frontend

When a record in sys_file_reference is created the sys_language_uid from the last language of the record (before saving) will use and never change in future. I can change the language in the news record (and saving) but in sys_file_reference data the sys_language_uid is ever the same.

The same behavior is for sys_file_reference entries using in tt_content. But i think the frontend rendering is other. There is no problem with this.

For Extbase the sys_language_uid in sys_file_reference must be checked whenever the langugae of the referenced data is changed. Or rather checking the language of referenced data at the moment of saving.

I hope this helps to fix the problems with this patch (d39c8b2d).

Am I correct that this is also related?:

<f:link.page pageUid="{dokument.originalResource.publicUrl}">Download</f:link.page>

returns a link to the file of the default language record - instead of the one that was indicated in the localised record.

The TCA entry for this:

        'dokument' => array(
                'exclude' => 0,
                'label' => 'LLL:EXT:publikationen/Resources/Private/Language/locallang_db.xlf:tx_publikationen_domain_model_publikation.dokument',
                'config' => \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::getFileFieldTCAConfig('dokument', array(
                        'appearance' => array(
                                'createNewRelationLinkTitle' => 'LLL:EXT:cms/locallang_ttc.xlf:images.addFileReference'
                        ),
                ), $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext']),
        ),

I think so. But i don't checked this ;-). In news extension it helps to change the TCA for fal_related_files or fal_media.

https://forge.typo3.org/issues/57021

When i'm using the localize buttons in backend it's working fine. For records without default language it helps to localize the files before using.

@Karsten

I tried adding

                                'showSynchronizationLink' => 1,
                                'showAllLocalizationLink' => 1,
                                'showPossibleLocalizationRecords' => 1

to "appearance" in the code above, and then I was able to do some "localizing" in the backend, but none that made sense to me - and no effect in the frontend.

@Urs

I've tested again and here are my results:

1. news in default language with related file, translate this news and localizing the related file

• in frontend the file with path and title from the default language is shown
• debug {relatedFile} in fluid template and change {relatedFile.originalResource.title} -> {relatedFile.title}, then the translated tile is shown but with the path to the same file like default language. Of course that's right, only the metadata were localized. Ok, another file is needed. But when i add the new file (localized before), there is no file in frontend. You're right.

2. single translated news without a parent record in default language

• here i add a related file (localized before) and everything is fine frontend. I add a new file (localized before too) again and this is also shown in frontend

Mmmh.., crazy. In translated news it doesn't work but single news in translation works fine. The same behavior is with images.
Only for same files/media in default language and translated the localize buttons serve his functions.
When different files or media in localized news and in default language are needed, it seems to me it's required to use single news records in the relevant language.

I'm using sys_language_mode=strict and TYPO3 6.2.6

This bug is also related:
Bug #63092

It is related to others regarding wrong sys_language but this one deals with the fact that I change the language AFTER I once saved the record and had FAL/IRRE records related to it (will not update language).

Furthermore the creation of IRRE related records does only updated default language, if the language is "> 0" but this ignores all language option (-1)

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35486

I have an error in frontend if I use this patch.

#1: PHP Warning: Missing argument 4 for TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper::getPreparedQuery(), called in /html/typo3/typo3_src-.2.9/typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapper.php on line 332 and defined in /html/typo3/typo3_src-.2.9/typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapper.php line 345

TYPO3 6.2.9

Could you please check your TYPO3-sources? That said function only has 3 arguments (typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapper.php).

I added a defaultvalue to $forceTranslationOverlay in method getPreparedQuery in file typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapper.php and now the patch works.

    protected function getPreparedQuery(\TYPO3\CMS\Extbase\DomainObject\DomainObjectInterface $parentObject, $propertyName, $fieldValue = '', $forceTranslationOverlay = FALSE)

With the defaultvalue for $forceTranslationOverlay patch works for me.

With the defaultvalue for $forceTranslationOverlay the patch works for me too.
Only in translated news without l18n_parent in default language I need to localize the image or files before. Otherwise they are not shown in frontend.

I made a small extension in correspondence with Stefan Frömken to override the classes of the patch as a workaround:

https://bitbucket.org/t3easy_de/faltranslation

If you extend \TYPO3\CMS\Extbase\Domain\Model\FileReference in your own extension you have to override getOriginalResource in your extended model or extend \T3easy\Faltranslation\Domain\Model\FileReference.

Thanks a lot for your extension. It helps a lot localized datasets.
Unfortunatly the image is missing if you are in not default language and the dataset is of language type -1 "All languages".

Hey,

will there be a solution in the near future? Or is there a simple workaround. I currently develop an extension where the path to the files has to be different in any language.

Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35486

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32718

Patch does not work for dataset of language type -1 "All languages"

Hi, I did test the extension of Jan Kiesewetter (thanks a lot!!!) and it works well in TYPO3 6.2.13-dev.

Here http://www.csthermos.it/ you can find a working example.

All those tiles in the homepage are rendered by an Extbase plugin set with Language = All.
This plugin uses one record from the model "Mosaic" set with Language = All from a storage page.
Finally this record has MM relations with all those records of model Tile.
Those tiles are in default language but are also localized in french http://www.csthermos.it/fr/
One tile "SOCIÉTÉ" has a localized image (you can see the french payoff of the logo)
I've tested also a localized tile without image and it correctly uses the default language tile image.

The only problem is that the french shows also a not localized tile "PETIZIONE IVA 22%" even if I set sys_language_overlay = hideNonTranslated (that instead works with tt_content records) but I guess that's another bug.

Another example from this website it's a product detail www.csthermos.it/prd/stufa-pelletbiomassa/sintesi-90/
if you scroll to the bottom to the DOWNLOAD section you can find a pdf catalogue in italian and switching to the french there's the french version. So it's all ok.

Tested with:
sys_language_mode = content_fallback ; 1,0
sys_language_overlay = hideNonTranslated
and
sys_language_overlay = 1

I hope that this will be fixed in 6.2 LTS because otherwise it's very very hard to handle multi-language sites that use Extbase.

Is there a chance to get the patch into core?
As Christian said, it's really hard to handle multi-language sites. Including this patch will help a lot. Even if language "all" is not supported.

Jeb, this patch is working great - besides the known issue with language "all".
But I agree with Heiko, this would be a smaller issue than the current situation.