Schlagwort-Archiv: typo3.surf

T3DD13 Deployment Workshop – EXT:coreapi – (Part 4/10)

This article is part of a series about my deployment workshop on the T3DD13. Make sure to read the other posts, too.

We all love the CLI, don’t we?

While administrating TYPO3 CMS instances, you quickly stumble over common tasks, like clearing the caches, you only able to do by open up a browser, login in the backend and do it there. This might be no problem for one single instance or small projects, but if you want to execute these tasks in a automated deployment, you need a new way to run them. You need a way to do it from the CLI.

There are some extensions, which address this issue and solve it for single tasks: For example EXT:cleartypo3cache provides a CLI command to clear the caches and EXT:t3deploy enables you to run a database compare from CLI, too.

EXT:coreapi

When I started to think about automated deployments for TYPO3 CMS, the idea of EXT:coreapi raised up. How awesome, if i (as a developer) could do anything using the CLI instead of need to do it in the browser…

EXT:coreapi is supposed to provide an easy to use API for the „everything“ one might want to automate or just like do it from CLI, but is only able to do using the backend before. It abstracts the internal TYPO3 APIs, which might differ in major TYPO3 versions (4.5, 6.x, …). EXT:coreapi implements and offers service classes, which can be used in your own extension (e.g. to build a custom webservice). But most important, it provides commands to run from CLI, too.

Besides of the probably most needed „database compare“ and several „clear cache“ commands (clear all, clear pages cache, clear configuration cache) it offers commands to handle extensions (info, listInstalled, updateList from TER, fetch an extension from TER, import an extension, install / uninstall extension, create upload folders, configure extension).

plans

It’s also planed to provide more commands/APIs, like

  • managing backend users (list, create, update, delete)
  • printing/output the page tree or sub trees
  • a generic data API to list/create/update/delete any records by using the internal DataHandler (aka tcemain) of TYPO3
  • run and check reports from the reports module
  • list, get and set TYPO3 configurations (e.g. TYPO3CONFVARS)
  • create database dumps (and excluding all caches tables and e.g. sys_log if you want)
  • im-/export .t3d files
  • output/analyze TypoScript for a certain page (like the Template Object Browser in the Backend)

During the workshop on the T3DD13 a small group formed and discussed, what’s missing and can be improved. You will find a protocol here.

EXT:coreapi is available on github and we collect and discuss the ideas, and feature requests there. Feel free to join the discussions in the issues on github and fork the project to solve some of the issues. I’m really looking forward to your pull requests.

T3DD13 Deployment Workshop – TYPO3 Surf – (Part 3/10)

This article is part of a series about my deployment workshop on the T3DD13. Make sure to read the other posts, too.

TYPO3.Surf

Surf is an open source tool for deployment, just like Capistrano, Phing or Ant. It’s based on the application framework TYPO3 Flow and thus written in PHP. Even the deployment configurations are written in PHP, so no hassle with ruby scripts or large XML files. Surf can easily be extended by your own Flow package, to add new tasks or even custom deployment workflows. For details about how Surf and the deployment configurations are working, check my article here.

Because Surf is a package for TYPO3 Flow, it can be used „embedded“ to deploy your Flow-based application. You just need to add the TYPO3.Surf package to our application distribution to enable it to deploy „itself“. But you can also setup a „standalone“ Surf instance, which can take care of all your deployments, even if the are not Flow-based. Actually you can deploy anything, which is versionized (currently only Git is supported out of the box, but it easy to add an SVN Checkout task). So it’s no problem to deploy e.g. a TYPO3 CMS or a WordPress, too.

Even if TYPO3.Surf is already used in several production setups, it’s still beta. Some days ago, I met Christopher Hlubek in Kiel at networkteam. We discussed what is missing and needs to be done for a first TYPO3.Surf 1.0 release. There are some easy tasks (like adding a SVN checkout task, some other new tasks or some API cleanups) as well as some meta tasks (seting a product website; provide sample configurations/recipes). I will create a bunch of new issues and cleanup the bugtracker within the next days on forge. If you like to contribute and work on one of these tasks, please get in touch with us!

Custom Surf tasks

During the workshop at the T3DD, we discussed, which custom tasks the participants, which already use Surf, created for theri own needs.

Martin, from n@work, told us about a „reload Apache“-task, which was needed, because the customer virtualhost configuration was versionized in the git repository, too. Beside of that, they created tasks for EXT:coreapi and EXT:t3deploy to clear the caches of a TYPO3 CMS instance or run the database compare tool automatically.

Felix, from Lightwerk, shared some tasks e.g. to create or apply MySQL-dumps, which is pretty helpful, if you want to create for example a testing/dev environment based on the current production database. These tasks, among several other changes, are currently pending for review in Gerrit.

Flexible transfer method

One bigger change in TYPO3 Surf, which was merged recently, are a set of changes bunched toghether under the topic „flexible transfer method“. Before TYPO3 Surf only supported Git to get the code to deploy and additionally the Git repository had to be available from the target node. Surf to connected via ssh to the target node and remotely executed the Git cli command to clone and checkout the repository there. This obviously can’t work, if you don’t have Git installed on the target node. Or, the Git repository might only be available from an internal network and is not reachable from the target node system. To cover these cases, Surf now provides two new stages in the SimpleWorkflow: package and transfer.

In the package stage, Surf can be configured to prepare the code/files locally instead of doing that on the remote host. Surf can do a local git clone, composer update or whatever is needed to prepare the code. Then, during the new transfer stage, the packaged code will be transferred to the remote nodes. This can be done by a rsync, but also with scp or even ftp (in the moment there is only a RsyncTask. But its pretty easy to implement the others, too). Using this new methods the nodes don’t need direct access to the repository anymore. As a side effect, when deploying in a multi-node-environment, your release/code only needs to be prepared/packaged once, instead of on each single node independently. This might speed up your deployment a little.

next steps

The next steps for TYPO3 Surf is to get it ready for a stable 1.0 release as soon as possible. To archive that we could use some help for implementing tasks, refactor some API methods and reviewing/testing changes. Also feedback of people/agencies using TYPO3 Surf to deploy whatever would be highly appreciated.