TYPO3 Flow Base DistributionPackagesApplications

Feature #37077

Surf should also support local checkout and rsync to server

Added by Thomas Allmer about 3 years ago. Updated about 2 years ago.

Status:Needs Feedback Start date:2012-05-10
Priority:Should have Due date:
Assigned To:Thomas Allmer % Done:

90%

Category:-
Target version:-

Description

For that to work the following steps should be done:
1) a possibility to checkout locally
2) a Rsync Task
3) move the "copy from cache to release" into a separate Task (CopyCacheToReleaseTask)

Details:
h2. 1) checkout locally
I currently have an option "$application->hasOption('localPath')", if present it well checkout on localhost to this path instead of the remote host and deployPath. Opinions?
I would probably also want to change the path "cache/localgitclone" to ".localclone"? As the folder for the "remote" should be the same regarding of usage as git or rsync. Or more meaningfull name? sugguestions?

2) Rsync Task
pretty straight forward rsync from local folder ".localclone"? to remote folder ".localclone"

3) CopyCacheToReleaseTask
As we now need to copy from a "cache or clone" in two separate task it would be better to create a task for it.
Furthermore it would give the possibility to add Task after this one instead of the gitcheckout.

Resume:
I currently have a working solution but I'm not sure about the naming and stuff so I ask here for opinions before creating a patch for gerrit. Furthermore I also have a completely separate Application "FLOW3rsynch" as it wasn't possible to use FLOW3 or BaseApplication. So I'm again not sure howto do this "more" properly...

Looking forward to your input :)

History

#1 Updated by Thomas Allmer about 3 years ago

sorry for the wrong formatting... can't change it :/

#2 Updated by Thomas Allmer about 3 years ago

ok while thinking again it might be good to change the
cache/localgitclone to something like .localclones/$deployment->getName()/
as you may use one Surf Instance to deploy different Installations/git repositories, which currently fails as once you change the git url it tries to update one git repository with a sha1 of another repository which obviously won't work.

For example TYPO3v4/v6 deployment may be using such an approach
http://forge.typo3.org/projects/extension-deployment/wiki

#3 Updated by Thomas Allmer about 3 years ago

  • % Done changed from 60 to 90

for the time being I have a fork here
https://github.com/daKmoR/TYPO3.Surf

If anyone from the Surf Team could give me some feedback if it's done right and it could go into the "core" I'm more than willing to provide "isolated" reviews for certain features.

it "extends" TYPO3.Surf with the following features
- BaseApplication
  • define "transferMethod" [currently only rsync, if undefined git (like before) is used]
  • copycachetorelease Task
- TYPO3v6
  • createdirectories
  • symlinkdata
  • symlinkcore
  • copyconfiguration

also it fixes some stuff like:
- do git checkout recursively
- own git checkout folder for each deployment (allows to deploy multiple installations)
- use expect script also for rsync, scp... (so you aren't asked for any password)

#4 Updated by Tobias Liebig about 3 years ago

Re: 1) checkout locally

did you try using "localhost" as hostname? If the host is "localhost", no remote/ssh commands will be executed, but everything is done locally instead. (see TYPO3\Surf\Domain\Service\ShellCommandService)

#5 Updated by Thomas Allmer about 3 years ago

hey Tobi,

if you take a look at my last comment you can see that I add an "option" for the Base Application. It's called "transferMethod" and it currently supports git and rsync. For that to work I change the git checkout task to accept an option "localcheckout" which will checkout the code into an local cache dir and then you can use whatever method you want to transfer it to the server.

Here is an example deploy script using the "transferMethod" option.

<?php
    $workflow = new \TYPO3\Surf\Domain\Model\SimpleWorkflow();
    $deployment->setWorkflow($workflow);

    $node = new \TYPO3\Surf\Domain\Model\Node('<domain>');
    $node->setHostname('<domain>');
    $node->setOption('username', '<username>');
    $node->setOption('password', '<password>');
    $node->setOption('port', '<port>');

    $application = new \TYPO3\Surf\Application\TYPO3v6();
    $application->setDeploymentPath('/path/to/your/public_html');
    $application->setOption('repositoryUrl', '<repositoryUrl>');
    $application->setOption('transferMethod', 'rsync'); //defaults to 'git', supports: [git, rsync]
    $application->setOption('keepReleases', 2);
    $application->addNode($node);

    $deployment->addApplication($application);
?>

#6 Updated by Rens Admiraal over 2 years ago

I had a look at the github fork and substracted the rsync command out of it in a separate package so we can discuss it during the code sprint we have this week.

https://github.com/radmiraal/Radmiraal.SurfJobs

#7 Updated by Thomas Allmer over 2 years ago

cool :)
If you have any more questions just let me know... I now use if for about 10 month with multiple TYPO3 Installations and different Servers (most without git support). We have done probably a few hundred live and dev deployments during this time... so for us it's a real time saver as you just configure it once and everyone on the team can use it.

#8 Updated by Tobias Liebig about 2 years ago

  • Status changed from New to Needs Feedback
  • Assigned To set to Thomas Allmer

Hej Thomas,

i think this issue is solved by the new "flexible transfer method", which we merged recently.
https://review.typo3.org/#/q/status:merged+project:Packages/TYPO3.Surf+branch:master+topic:flexible-transfer-update,n,z

Could you check that and then close this issue? Thanks.

Also available in: Atom PDF