Schlagwort-Archiv: deploy

T3DD13 Deployment Workshop – EXT:migrations – (Part 6/10)

This article is part of a series about my deployment workshop on the T3DD13. Make sure to read the other posts, too.

Idea of „migrations“ for more then database schema updates

This basic concept of „migrations“ for TYPO3 CMS goes back to a talk on the T3DD in 2009 I held together with Christopher. Back then we implemented a Proof of Concept for a customer project and presented that on the devdays.

In that project we had multiple installations (like the live system, a staging sytem, and several developer systems) and needed to apply the same changes to all those independent systems. Those „changes“ were things like templavoila TO/DS and mappings, setting up pages with configured plugins, application related configurations in the database and so on. Basically everything one need to do in the backend.

We started to maintain a wiki page and collected all changes (as a step-by-step guide) which need to be done on the next release. Whenever someone updates the codebase on one of the systems, he needs to check the wiki what needs to be done in the backend. This approach was very time-consuming and error-prone. So we wanted to be able to script all these changes and deliver them with the code base in the project SVN (we didn’t used Git back then) and be able to track which changes are already done on one system and which are missing.

From Ruby or Doctrine you might know the concept of migrations. It’s basically made for database schema updates, like creating a new table, adding columns or updating column definitions. A migration provides update from one stage of the database to a new one. A migration can be „applied“ („up“ or „execute“) or „removed“ („down“ or „rollback“). Each installation „knows“, which migrations are already applied and which are not. With a simple command you can apply all missing migrations (in the correct order) to make sure your database is up to date.

In TYPO3 we have a solution for database schema updates already: The „database compare tool“ compares the current database table definitions with those described in the ext_tables.sql files in the extensions. But we can make use of this basic concept to solve the issues described above.

A migration is basically a PHP class with a certain naming convention and in a defined folder. As it is just a file, it can be versionized and distributed to all instances using the VCS (e.g. Git or SVN). Each migration class has a „up“, a „down“ method and a unique identifier (a UUID, a timestamp or something else).

Each instance knows which migrations are available (by scanning the defined folders for migration class files) and which of them are already applied. In a backend module and by using a CLI command the user can list and run the missing migrations. This will call the „up“ method for each migration and memorize all successfully applied migrations.

As the migrations as just PHP classes, you can do everything you need within the „up“ (and the „down“) methods. For example it can use the service classes provided by EXT:coreapi to create/update records.

I see a lot of use cases for such a tool and I’m sure it will help a lot to solve the issue of not-versionized configuration and application related data, which are in the database of TYPO3, but not in files currently.

EXT:migrations

All described above is just ideas and concept. There is no implementation yet. I’m willing to work on a proof of concept and a first implementation. If you like the idea, if you need something like that or have any suggestions, please get in touch with me!

T3DD13 Deployment Workshop – EXT:coreapi – (Part 4/10)

This article is part of a series about my deployment workshop on the T3DD13. Make sure to read the other posts, too.

We all love the CLI, don’t we?

While administrating TYPO3 CMS instances, you quickly stumble over common tasks, like clearing the caches, you only able to do by open up a browser, login in the backend and do it there. This might be no problem for one single instance or small projects, but if you want to execute these tasks in a automated deployment, you need a new way to run them. You need a way to do it from the CLI.

There are some extensions, which address this issue and solve it for single tasks: For example EXT:cleartypo3cache provides a CLI command to clear the caches and EXT:t3deploy enables you to run a database compare from CLI, too.

EXT:coreapi

When I started to think about automated deployments for TYPO3 CMS, the idea of EXT:coreapi raised up. How awesome, if i (as a developer) could do anything using the CLI instead of need to do it in the browser…

EXT:coreapi is supposed to provide an easy to use API for the „everything“ one might want to automate or just like do it from CLI, but is only able to do using the backend before. It abstracts the internal TYPO3 APIs, which might differ in major TYPO3 versions (4.5, 6.x, …). EXT:coreapi implements and offers service classes, which can be used in your own extension (e.g. to build a custom webservice). But most important, it provides commands to run from CLI, too.

Besides of the probably most needed „database compare“ and several „clear cache“ commands (clear all, clear pages cache, clear configuration cache) it offers commands to handle extensions (info, listInstalled, updateList from TER, fetch an extension from TER, import an extension, install / uninstall extension, create upload folders, configure extension).

plans

It’s also planed to provide more commands/APIs, like

  • managing backend users (list, create, update, delete)
  • printing/output the page tree or sub trees
  • a generic data API to list/create/update/delete any records by using the internal DataHandler (aka tcemain) of TYPO3
  • run and check reports from the reports module
  • list, get and set TYPO3 configurations (e.g. TYPO3CONFVARS)
  • create database dumps (and excluding all caches tables and e.g. sys_log if you want)
  • im-/export .t3d files
  • output/analyze TypoScript for a certain page (like the Template Object Browser in the Backend)

During the workshop on the T3DD13 a small group formed and discussed, what’s missing and can be improved. You will find a protocol here.

EXT:coreapi is available on github and we collect and discuss the ideas, and feature requests there. Feel free to join the discussions in the issues on github and fork the project to solve some of the issues. I’m really looking forward to your pull requests.