Saturday, September 6, 2014

Integration of RabbitMQ in Zend Framework 2

When it comes to asynchronous processing in php, there is a robust way to do it, with RabbitMQ. Using RabbitMQ (an implementation of the AMQP protocal with some nice additional features) is pretty straightforward using the PHP AMQP Extension. What was missing so far was a nice Zend Framework 2 Integration, a generic consumer implementation, a simple RPC-client & -server implementation.

The producer interface is very simple:

On the other hand we have an extensable consumer interface:

The consumer supports the following:

- defer message for later decision whether to ack or nack
- ack & nack blocks of messages
- configure prefetch count
- automatic exchange and queue creation
- set callback for handling delivery of messages
- set callback for handling flush deferred messages
- set error callback
- set configurable logger
- idle timeout
- non-blocking
- configurable wait timeout
- unix signal handler support
- generic cunsumer controller and cli integration
- supervisor support
- integration in HumusSupervisorModule

Even more to come....

There is also a manual at Read the Docs and a Demo-Module incl. RPC-Client & -Server Example. Thew manual and the demo module will help you getting startet with Queuing in Zend Framework 2 without any pre-existing knowledge and guide you how to create topic-exchanges, header-exchanges, fanout-exchagesn, how to set up dead-lettering, message timeout, and so on.

Your contributions are welcome!

Thursday, July 18, 2013

Minifesto - MANIFESTO FOR MINIMALIST SOFTWARE ENGINEERS

  • Fight for Pareto's law, look for the 20% of effort that will give you the 80% of results.
  • Prioritize, minimalism isn't about not doing things but about focusing first in the important.
  • Perfect is enemy of good, first do it, then do it right, then do it better.
  • Kill the baby, don't be afraid of starting all over again. Fail soon, learn fast.
  • Add value. Think constantly how you can help your team and position yourself in that field/skill.
  • Basics, first. Follow always a top-down thinking starting by the best-practises of CS.
  • Think different. Simple is harder than complex, which means you'll need to use your creativity.
  • Synthesis is the key of communication. We have to write code for humans not machines.
  • Keep it plain. Try to keep your designs with few layers of indirection.
  • Clean kipple and redundancy. Minimalism is all about removing distractions.

I am currently working on the german translation and will get the russian translation from a friend, too.

Sunday, February 24, 2013

Sending stream responses with Zend Framework 2, Part 1

Sending stream responses is a new feature in Zend Framework 2.1. This short introduction will help you getting startet on how to send stream responses in Zend Framework 2.
Most times an application would deliver file downloads or video streaming with static assets, delivered directly frm Apache or Nginx. But sometimes there are restrictions: Only logged-in users should be able to download this file, watch this content, perhabs even a more complicated permission system. Then the application needs to do some permission checks and stream the response.
In this example we assume a simple file download, we deliver the file as stream response, because this is much less memory consuming and more efficient, compared to grabbing the file content and pushing it to the response body of the default response object.

Even more complex use cases with download resume and speed limits are also possible in the future, but this is part of my next blog post about this topic. However you can see how easy it is to send stream responses with Zend Framework 2. As we return the response object directly in the controller, no view is created and rendered, too, so this is really a very fast operation.

Friday, February 1, 2013

List of current php version in major linux distros

I tried to make a list of current php versions in major linux distributions. Please feel free to correct or add additional one per comment.

Distribution PHP Version
Ubuntu 14.04 LTS 5.5.9
Ubuntu 12.04 LTS 5.3.10
Ubuntu 12.10 5.4.6
Debian 7.5 Wheezy 5.4.4
Debian 6.0.6 Squeeze 5.3.3
Cent OS 6.3 5.3.3
Arch Linux 5.4.11
Fedora 18 5.4.9
Fedora 17 5.4.1
Mandriva 2011 5.3.20
Mandriva 2010 5.3.2
Mageia 2 5.3.19
OpenSUSE 12.2 5.3.15
OpenSUSE 13.1 5.4.20
Slackware 14 5.4.7
PCLinuxOS 5.3.8
Red Hat (Enterprise) Linux 6.3 5.3.3
Red Hat (Enterprise) Linux 7.0 RC 5.4.16
SUSE Linux Enterprise System 11 SP1 5.2.6
SUSE Linux Enterprise System 11 SP2 5.2.14
SUSE Linux Enterprise System 11 SP3 5.3.17

Wednesday, January 9, 2013

Using git-svn for projects with many branches

About branches: Git-svn would normally clone all revisions from all branches by default. This is a time-consuming process, so I strongly recommend adding only those branches, that you need for your work.
We don't specify a branches section in our git configuration, instead we are attaching branches as new remotes. That also means, that we can't create or delete branches on a svn server, they have to be created with the corresponding svn commands. 1) Cloning the repository, checking out trunk

First create a directory where you want your git repository to be on your harddrive, than initialize a new git repository. Next you have to edit your git config and add the trunk as svn-remote.

edit .git/config and add: Next you clone the repository. This will check all SVN revisions and start a download for every revision of every file of the trunk, so this will take a while. You end up with a local branch "master" that tracks the svn remote "trunk". Type "git svn info" to show information. 2) Keeping things up to date

You should never work on the master branch, that one is only reflecting the trunk, it's a mirror, so please never change files here. Just update the master to reflect the trunk, that's all. The --ff flag forces a fast-forward.

3) Creating new local branches and rebase

Sometimes, you just want to create a new local branch and work on it. (Perhabs a small bugfix branch, that immediatly gets merged into trunk.) ... do some work … All unit tests are good, and you want to push that branch perhabs directory to svn trunk. But first, we need to rebase. Someone else could have made commits to trunk in the meantime, so you have to apply his changes first, than yours, that means rewriting your local history. .. now you have the choice, you can pick, reword, edit, squash and fix commits … .. it's always recommended to make a dry-run first, so you don't push to the wrong location accidentialy 4) Adding a new svn remote branch

Because we have so many branches, that git would force you to download them all, we just specify the explicit branches we want to track. I recommend that you specify the revision number here, otherwise git will download every revision, starting with 0 first. If your branch was created at revision 100000, you will have to wait a long time. 5) Reintegrating a single commit into trunk

In the most easiest case, you have a branch with just one single commit, so you can just cherry-pick it. Output: As you can see, the first commit is "c56d7da69e96a662aa9aad8cd394a381bad64807", but that one is just the svn copy, we don't want to merge this, so only "2db1182ddc0e85b3a9c468bd7610a58c3bb99d0a" is interessting. Lets change to master, update, run the unit tests and commit. ... run unit tests

6) Reintegrate a branch into trunk Your editor should pop up with something like this in it: Note, that we don't want to merge "0c70bbe", because that would was the svn cp command. Just delete this line. Save the file, and exit. You can even squash commit or edit commit messages. Run the unit tests again. 7) Delete the branch checkout

The following command sequence will delete (locally only) the SVN branch and its history: To remove the branch on the svn server, remove it with svn rm <branchurl>.

8) Converting a local branch into a remote branch

Basically all it amounts to is a rebase, with some svn trickery.

Next you add that branch to your git config.

Verify

Friday, November 16, 2012

How to test a Zend Framework 2 Application

A Zend Framework 2 application has a couple of test suites. First of all, it contains Zend Framework 2 including its test suite. Then you have also your own module (or modules) which can contain its own test suites. Last but not least, you could have additional thirdparty modules (f.e. DoctrineORMModule or DoctrineModule) and thirdparty libraries (f.e. Symfony\Console, Doctrine\ORM, Doctrine\Common), which contain its own test suites.
To run all test suites for your application, you need to install phpunit and it's dependencies, then you need to run something like this:

Well, of course you can create a simple bash script, that does the job for you. But if you want to exclude disable modules, that won't work. You have to change the script each time, you disable or install a module.

The solution: Humus PHPUnit Module

Humus PHPUnit Module is a Module for Zend Framework 2 for unit testing. It is able to test all your zf2 modules and libraries at once.

The installation is quite easy:


And you DON'T have to have phpunit installed at all! HumusPHPUnitModule will install EHER/PHPUnit through composer and you have phpunit installed in your vendor directory.

Usage is easy, too:

How it works:
You have two possibilities, to enable a test suite for the humus phpunit module.
1) Using HumusPHPUnitModule\ModuleManager\Feature\PHPUnitProviderInterface

You can implement the PHPUnitProviderInterface in your own modules and provide an array of phpunit.xml files. The PHPUnit Runner collects them and runs all of them. To run the tests for thirdparty modules, that are independent from HumusPHPUnitModule, you can either write your own module, that returns phpunit.xml files for every thirdparty module you have (this can be one module per thirdparty-lib, but also a single one returning an array including all phpunit.xml files for all your thirdparty modules. Another way is using the configuration file.

2) Using configuration

Just put this in your application configuration and run the bin script (./vendor/bin/humusphpunit).
That's all, there is also a HumusPHPUnitSampleApp - This is a demo application to show the features of Humus PHPUnit Module. It tests a sample application module with the PHPUnitListener and the Doctrine\Common lib and Zend Framework 2 with configuration.

Friday, October 12, 2012

When migration to Zend Framework 2 seems impossible

The problem

When you got a really big codebase and lots of developers working on it, releasing new features and bugfixes every 2-3 weeks, it seems impossible to migrate to Zend Framework 2, soon.
That's because you can't stop further development, you have customers waiting for the new features, they wait for bugfix releases, to get stuff running, that's buggy, you have more than 1mio lines of code and possibly > 100 active developers on that codebase. How can you migrate? I stombled accross this problem already when the first ZF2 betas came out and startet to write some code that should solve the problem.
The result is HumusMvc, it integrates Zend Framework 2's ModuleManager and ServiceManager in a ZF1 application.

The migration path

First of all, you create a new skeleton for you application. Simply clone the HumusMvcSkeletonApplication and use it as base for you skeleton app.
At the next step, you create a repository per module you have in your old ZF1 application. Put the source code in the "src/" directory of that module, create a Module.php file, name the Module, put the module configuration in it, etc. - in other words, create a simple ZF2 module out of your ZF1 application module. If you use HumusMvcAssetManager, too, you can also provide a public directory in your module and use the asset manger. That way you can override assets in a very similar way you override view scripts in a ZF1 application.

The module now can look something like this:
- MyAppDefaultModule/
    - config/
        - module.config.php
    - layouts/
    - public/
        - css/
        - js/
        - img/
    - src/
        - controllers/
        - models/
        - views/
    - tests/
    - autoload_classmap.php
    - composer.json
    - Module.php

If you use ZF1 with the Resource-Autoloader, than you don't have psr-0 compliant classnames, so define your autoloading simply with classmap autoloader.
In order to install your module, just require it with composer.

In your module.config.php you provide your front_controller configuration, you have to tell the front controller, that you have a module, where it can dispatch.



You can now define some resources with ZF2's service manager in your module config file. You can remove those resources from bootstrapping later. HumusMvc ships with Navigation, Translator and Locale factories, so you don't need to handle these, just configure them.
If you have something like a db connection, that was bootstrapped in your old application, you need to do this again. Therefore you use the "onBootstrap" event. Well, you could define everything right now with the service manager, but that is exactly that much work, we wanted not to do now, because that would stop the further development for quite a time.

MyAppDefaultModule\Module.php
Here is a short example you to bootstrap db connection on every request and put the application configuration into Zend_Registry.

Further refactorings

At first, I recommend to remove all "onBootstrap" methods when neccessary and put this stuff in your service manager, so you only instantiate the objects, when they are need, not on every request.
In the next refactoring rounds, you can refactor only one module at a time, so development in the other modules can still be done. Put all stuff in the service manager and use it as a service locator inside your application. Remove Zend_Registry everywhere. When you are done with that, use a php namespacer tool (or do it by manually) and refactor to a namespaced codebase. Remember, code is per module, so you can do this step module per module. Further bugfixing and development of new features is still possible.

Final migration to Zend Framework 2

You should now have a very flexible and modern Zend Framework 1 application, that makes use of a lot of stuff from Zend Framework 2, already: Service Manager, Module Manager, Event Manager, just to name a few. You don't have any expensive upfront bootstrapping any more, you moved everything to the service manger. You should have a namespaced codebase. You use ZF 2 classes much more, than ZF1 classes. - Now it's time to do the final migration.

Refactor all controllers, refactor route configuration refactor all view scripts, and so on. Now it's time for the hard work, we skipped before - but we had a couple of month time to bring our codebase to something, that _can_ be refactored to a ZF2 codebase very fast.