Use Phing to update your SVN-version-controlled website automatically, through FTP

If you’re working on a PHP project, like with any other project, probably comes a time when you need to “build a new release,” e.g. update the production web site with the latest version of the code. And doing the whole thing manually isn’t the most efficient way to get things done, especially if you’re lazy, or have to do that every once in a while, over a long period of time.

There are a few softwares out there to take care of that to-do list for you, one of them, running on PHP, is called Phing, for PHing Is Not GNU make. It is a PHP build tool based on Apache Ant, and it will allow you to save lots of time. It’s especially true if you use SVN or Git, since both are supported; Support for other VCS, such as Mercurial, are planned, and for those who cannot wait, the project is meant to be easily extended.

Imagine you’re working on a PHP project (e.g. a web site), with subversion as version control system, but where the site needs to be updated manually through FTP. In such a scenario, here’s what your to-do list, for whenever it comes time to “build a new release” and update the site, might look like:

1. Create a new tag for the new version
2. Make sure temp folder is empty
3. Export all new/modified files since last version/tag in the temp folder
4. Upload (FTP) said files
5. Clear temp folder

Obviously if updating the site itself could be done without FTP but through a simple svn switch, things would be easier. Still, Phing would allow to make things easier just as nicely. It could for example still be used to create the new tag, trigger the switch, etc

That is, either way, here’s what your Phing-powered to-do list might look like:

1. Run Phing

Interested?

Introduction and quick example

Phing is really easy to install through PEAR, and (might) require a few other components to make things work, based in what tasks you’ll want to have done. For instance, it relies on Net_FTP for FTP handling, or VersionControl_SVN for all SVN tasks. The later one happen to still be in alpha release, so you might have to specify the version number to pear if you want to install it, e.g: pear install VersionControl_SVN-0.3.4 (as time of writing)

Phing uses XML file for build files, and PHP classes for the different tasks it can accomplish, making it very easy to set it up, and (especially if you know PHP) extend it if necessary. Quick example, here’s what a simple build file looks like :

<?xml version="1.0"?>
<project name="foobar" default="build">
 
	<property name="svnpath"		value="C:\Programs\Svn\bin\svn.exe" />
	<property name="username"	value="username" />
	<property name="repo"		value="http://repo.url.here/" />
	<property name="livedir"		value="D:\Project\live" />

	<target name="password">
		<propertyprompt
			propertyName		= "password"
			promptText			= "Enter svn password"
		/>
		<if>
			<equals arg1="${password}" arg2="" trim="true" />
			<then>
				<fail message="Password required" />
			</then>
		</if>
	</target>

	<target name="svnswitch" depends="password">
		<svnswitch
			svnpath				= "${svnpath}"
			username			= "${username}"
			password			= "${password}"
			nocache				= "true"
			repositoryurl		= "${repo}"
			todir				= "${livedir}"
		/>
	</target>

	<target name="build" depends="svnswitch" />
</project>

Simple enough. As you can see, you can use properties, which can be defined in the XML file, come from user input (through task “PropertyPrompt”) or from other tasks. It also supports conditions, allowing you to e.g. run certain tasks based on conditions/properties from other tasks, etc (e.g. here we fail the build if no password was entered)

Tasks are grouped into “targets” which can also be depending on other targets to have been executed first. In the example above, when Phing is asked to process the target “build” (indicated per the “default” attribute of the project) it’ll work out that it depends on target “svnswitch” which itself depends on “password” and, therefore, will process “password” first, then “svnswitch” and then finally “build” (which, in our case, does nothing in itself)

Note: A target can have more than one dependencies, simply separate them using coma, e.g. "dep1,dep2,dep3"

Default SVN tasks are good, but might not be enough

By default Phing comes with a few SVN-related tasks, but unfortunately there’s nothing to do what we’re after, namely to export the differences between two tags. I guess you can’t really blame it on Phing, since it’s not supported by SVN either (and Phing’s SVN-tasks, or VersionControl_SVN, only interface SVN’s command-line), and needs a bit of extra work; Extra work that is often found in UIs or clients, such as TortoiseSVN.

Because we need that functionality, to upload (FTP) only new/modified files, and not the whole site again each time (or rely on the FTP client having to check every single file to determine which to upload), we’ll have to create a new task to fill our needs: “SvnExportDiff”

As I explained, the way things work is we’ll create a new tag, and then need the new/modified files (from the previous tag) to be uploaded through FTP. Creating the new tag can easily be done through Phing and its task “SvnCopy,” but what about the previous tag, the one used as origin point in the comparison?

Surely we don’t want to have to remember it, or have to manually go get its name or revision number. No, ideally Phing must be able to determine it itself, and so before looking into “SvnExportDiff” there is another new task to be created, one that will get the list of folders under folder “tags” (i.e. list of existing tags) and return the last one.

Introducing “SvnLastFolder”

A pretty simple task that will run svn’s command “list,” sort everything by name and set in specified properties its name and revision number, which can then be used with other tasks.

<?php

require_once 'phing/Task.php';
require_once 'phing/tasks/ext/svn/SvnBaseTask.php';

/**
 * Stores the last folder of a workingcopy in a property
 */
class SvnLastFolderTask extends SvnBaseTask
{
    private $namePropertyName = 'svn.last.name';
    private $revPropertyName = 'svn.last.rev';

    /**
     * Sets the name of the property to use
     */
    function setNamePropertyName($propertyName)
    {
        $this->namePropertyName = $propertyName;
    }

    /**
     * Returns the name of the property to use
     */
    function getNamePropertyName()
    {
        return $this->namePropertyName;
    }

    /**
     * Sets the name of the property to use
     */
    function setRevPropertyName($propertyName)
    {
        $this->revPropertyName = $propertyName;
    }

    /**
     * Returns the name of the property to use
     */
    function getRevPropertyName()
    {
        return $this->revPropertyName;
    }

    /**
     * The main entry point
     *
     * @throws BuildException
     */
    function main()
    {
        $this->setup('list');
    	
        $output = $this->run(array(), array('v'=>true));
        
        if (isset($output['.']))
        {
     		$a = array();
        	for ($i = 0, $count = count($output['.']['name']); $i < $count; ++$i)
        	{
        		if ($output['.']['name'][$i] != '.')
        		{
        			$a[$output['.']['name'][$i]] = $output['.']['revision'][$i];
        		}
        	}
        	krsort($a);
        	reset($a);
        	$this->project->setProperty($this->getNamePropertyName(), key($a));
        	$this->project->setProperty($this->getRevPropertyName(), current($a));
        }
    }
}

This is obviously a very simple thing, but when run on folder “tags” it is enough. And now, we have both information we need to extract difference between the two tags, this one and the one we’ll create using Phing.

Extracting differences between two tags

The task “SvnExportDiff” takes a repository URL and a revision number as parameters; It will then run svn’s command “diff” using those two parameters (as well as —summarize), then run svn’s command “export” for each of the files/folders either new (“A”) and modified (“M”) into the specified folder.

<?php

require_once 'phing/Task.php';
require_once 'phing/tasks/ext/svn/SvnBaseTask.php';

/**
 * Exports the differences between two revisions of a repository to a local directory
 */
class SvnExportDiffTask extends SvnBaseTask
{
    private $revision;

    /**
     * Sets the revision
     */
    function setRevision($revision)
    {
        $this->revision = $revision;
    }

    /**
     * Returns the revision
     */
    function getRevision()
    {
        return $this->revision;
    }

    /**
     * The main entry point
     *
     * @throws BuildException
     */
    function main()
    {
        $this->setup('diff');
        $output = $this->run(array(), array('summarize' => true, 'r' => $this->getRevision()));
        
        $repo = $this->getRepositoryUrl();
        $toDir = $this->getToDir();
        foreach (explode("\n", $output) as $line)
        {
        	if ($line[0] == 'A' || $line[0] == 'M')
        	{
        		// remove "line-header"
        		$line = substr($line, strpos($line, $repo));
        		// remove repo-url part
        		$a = explode($repo, $line);
        		$file = $a[1];
        		// extract path
        		$path = explode('/', $file);
        		array_pop($path);
        		// and make sure it exists. because we can get a file within a folder, and we'll export
        		// the file in a corresponding folder, only if it doesn't exists export will fail, so:
        		if (!file_exists($path = $toDir . implode('/', $path)))
        		{
        			mkdir($path);
        		}
        		// because the diff can get us both a folder and files within, with files first.
        		// so, we export if dest doesn't already exists, or is not a folder (which will
        		// fail unless force was true). Exists && folder was (probably) created when
        		// previously exporting a file within
        		if (!file_exists($dest = $toDir . $file) || !is_dir($dest))
        		{
	        		$this->log("Exporting $file from $repo to $toDir");
	        		$this->setRepositoryUrl($repo . $file);
	        		// need to re-do setup each time, since we have a new repositoryUrl
	        		$this->setup('export');
	        		$this->run(array($dest));
        		}
        	}
        }
    }
}

And there you have it, a task to export differences between the two tags. Now all it takes is to use Phing’s task “FtpDeploy” to upload all that to the production server, and the site has been “auto-updated.”

Our complete build file

What’s left to be done, such as creating/clearing the temp folder, can be done with Phing already. We can even use the task “PhpEval” to split the tag’s name into a prefix and a version number, and even set a default value. That task can evaluate a PHP expression, and the way it works is it will create and run through eval() a PHP code made of the argument “expression” with prefix “$retval = " (and, if needed, a semi-colon at the end) So when we need to just ran a little bit of PHP code, we can "abuse" this instead of having to create specific task.

To use your own tasks, you simply need to use the task “TaskDef” to define them in your project. In the following example, the previously described tasked have been put alongside the original ones, so in the same folder. You cann find the folder structure under the classname property, so you could just as well have them elsewhere (which would be better).

And here’s what we might end up with :

<?xml version="1.0" encoding="UTF-8"?>

<project name="foobar" default="build">
	<taskdef name="svnlastfolder"	classname="phing.tasks.ext.svn.SvnLastFolderTask" />
	<taskdef name="svnexportdiff"	classname="phing.tasks.ext.svn.SvnExportDiffTask" />

	<property name="tmpdir"			value="D:\Project\tmp" />
	<property name="svnpath"			value="C:\Programs\Svn\bin\svn.exe" />
	<property name="username"		value="username" />
	<property name="repo"			value="http://repo.url.here/" />
	<property name="ftpserver"		value="ftp.server.here" />
	<property name="ftpport"			value="21" />
	<property name="ftpusername"		value="username" />
	<property name="ftpdir"			value="/" />

	<target name="password">
		<propertyprompt
			propertyName		= "password"
			promptText			= "Enter svn password"
		/>
		<propertyprompt
			propertyName		= "ftppassword"
			promptText			= "Enter ftp password"
		/>
	</target>

	<target name="tagsvn" depends="password">
		<svnlastfolder
			svnpath				= "${svnpath}"
			username			= "${username}"
			password			= "${password}"
			nocache				= "true"
			repositoryurl		= "${repo}/tags"
			namepropertyname	= "svn.last.name"
			revpropertyname		= "svn.last.rev"
		/>
		<echo message="Last tag is ${svn.last.name} (revision ${svn.last.rev})" />

		<php
			expression			= "''; preg_match('/([^0-9.]*)([0-9.]+)/', '${svn.last.name}', $m); $retval = $m[1];"
			returnProperty		= "newtag.prefix"
		/>
		<php
			expression			= "substr('${svn.last.name}', strlen('${newtag.prefix}'));"
			returnProperty		= "newtag.version"
		/>

		<php
			expression			= "''; $a = explode('.', '${newtag.version}'); $last = array_pop($a); $a[] = ++$last; $retval = implode('.', $a);"
			returnProperty		= "newtag.version"
		/>

		<propertyprompt
			propertyName		= "newtag.version"
			defaultValue		= "${newtag.version}"
			promptText			= "New tag (from trunk) will be '${newtag.prefix}' with version"
		/>

		<svncopy
			svnpath				= "${svnpath}"
			username			= "${username}"
			password			= "${password}"
			nocache				= "true"
			repositoryurl		= "${repo}/trunk"
			todir				= "${repo}/tags/${newtag.prefix}${newtag.version}"
			message				= "tagging version ${newtag.version}"
		/>

		<echo message="New tag ${newtag.prefix}${newtag.version} created" />
	</target>

	<target name="cleantmp">
		<delete includeemptydirs="true" verbose="true">
			<fileset dir="${tmpdir}">
				<include name="**" />
			</fileset>
		</delete>
	</target>

	<target name="ftpdiff" depends="password,tagsvn,cleantmp">
		<svnexportdiff
			svnpath				= "${svnpath}"
			username			= "${username}"
			password			= "${password}"
			nocache				= "true"
			revision			= "${svn.last.rev}"
			repositoryurl		= "${repo}/tags/${newtag.prefix}${newtag.version}"
			todir				= "${tmpdir}"
		/>

		<ftpdeploy
			host				= "${ftpserver}"
			port				= "${ftpport}"
			username			= "${ftpusername}"
			password			= "${ftppassword}"
			dir					= "${ftpdir}"
			level				= "info"
		>
			<fileset dir="${tmpdir}">
				<include name="**" />
			</fileset>
		</ftpdeploy>

		<phingcall target="cleantmp" />
	</target>

	<target name="build" depends="ftpdiff" />
</project>

Now you have it; next time you’re ready to “build a new release” and update the site, all it will take is to run Phing. It will create a new tag (reminding you of the what last one is, and using it to come up with a default value (having bumped last part of version number) for the new tag’s name), clear the temp folder, export there all new/modified files, upload them through FTP, and clear the temp folder.

Notes

  1. thoomtech reblogged this from phpandme and added:
    Python’s Fabric at...my next project,
  2. phpandme posted this
Top of Page