Statically linked binaries with Haskell GHC

TL;DR Use AppImage

I am working on a nostr-project Futr written in Haskell. Since I don't want users of the application to install a bunch of dependencies manually upfront in order to use it, I digged deep into statically linking binaries with Haskell. I tried the nix build system, which didn't bring me to success, I only lost many many hours recompiling the same things again and again. My main requirements are secp256k1 (C-lib), glew and SDL2 (for monomer). What should have been "a not too hard task to solve" took me actually several weeks to find out. So I gave AppImage a try and it worked great basically from the first try on.

It's a two-step-process

1) Compile the haskell binary as usual
2) Build the AppImage

The resulting binary can then be shipped to the users. Compilation first (I'm using stack here): `stack build` - that part was easy!

First I need a build directory, my application is called "futr", so I name the directory "futr.AppDir". Within there I need two files:

"AppRun":

#!/bin/sh
SELF=$(readlink -f "$0")
HERE=${SELF%/*}
EXEC="${HERE}/usr/bin/futr"
exec "${EXEC}"

and "futr.desktop":

[Desktop Entry]
Name=futr
Exec=futr
Icon=futr-icon
Type=Application
Categories=Network
X-AppImage-Version=4702f2f

The latter is "only" for the desktop entry. Okay, so my binary requires some assets to live alongside the binary in the same directory, so let's add those:

cp -R assets futr.AppDir/usr/bin
cp assets/icons/futr-icon.png futr.AppDir

And all there is left is to create the final AppImage:

APPIMAGE_EXTRACT_AND_RUN=1 linuxdeploy-x86_64.AppImage --appdir futr.AppDir
ARCH=x86_64 appimagetool-x86_64.AppImage futr.AppDir

My project already uses Github Action, so I can automate the whole process and attach a binary to each release. Check the repository for the configuration: Futr Github Action

Sentry and amphp

Sentry is an Application Monitoring and Error Tracking Software. They provide an easy to use PHP SDK, but it is only useful for traditional web applications, not for asynchronous, non-blocking environments like in amphp. This is how I made is running for my specific use-case. It may not be a perfect solution and your use-case can be different.

First of all, I needed to rewrite a bunch of classes from the Sentry SDK (I am using ^3.0). Note that I don't implement their interfaces!

Client.php

<?php

use Amp\Promise;
use Jean85\PrettyVersions;
use Psr\Log\LoggerInterface;
use Psr\Log\NullLogger;
use Sentry\Event;
use Sentry\EventHint;
use Sentry\EventType;
use Sentry\ExceptionDataBag;
use Sentry\ExceptionMechanism;
use Sentry\Integration\IntegrationInterface;
use Sentry\Integration\IntegrationRegistry;
use Sentry\Options;
use Sentry\Serializer\RepresentationSerializer;
use Sentry\Serializer\RepresentationSerializerInterface;
use Sentry\Serializer\Serializer;
use Sentry\Serializer\SerializerInterface;
use Sentry\Severity;
use Sentry\StacktraceBuilder;
use Sentry\State\Scope;

class Client
{
    /**
     * The version of the protocol to communicate with the Sentry server.
     */
    public const PROTOCOL_VERSION = '7';

    /**
     * The identifier of the SDK.
     */
    public const SDK_IDENTIFIER = 'sentry.php';

    private Options $options;
    private Transport $transport;
    private LoggerInterface $logger;
    private array $integrations;
    private SerializerInterface $serializer;
    private RepresentationSerializerInterface $representationSerializer;
    private StacktraceBuilder $stacktraceBuilder;
    private string $sdkIdentifier;
    private string $sdkVersion;

    public function __construct(
        Options $options,
        Transport $transport,
        ?string $sdkIdentifier = null,
        ?string $sdkVersion = null,
        ?SerializerInterface $serializer = null,
        ?RepresentationSerializerInterface $representationSerializer = null,
        ?LoggerInterface $logger = null
    ) {
        $this->options = $options;
        $this->transport = $transport;
        $this->logger = $logger ?? new NullLogger();
        $this->integrations = IntegrationRegistry::getInstance()->setupIntegrations($options, $this->logger);
        $this->serializer = $serializer ?? new Serializer($this->options);
        $this->representationSerializer = $representationSerializer ?? new RepresentationSerializer($this->options);
        $this->stacktraceBuilder = new StacktraceBuilder($options, $this->representationSerializer);
        $this->sdkIdentifier = $sdkIdentifier ?? self::SDK_IDENTIFIER;
        $this->sdkVersion = $sdkVersion ?? PrettyVersions::getVersion(PrettyVersions::getRootPackageName())->getPrettyVersion();
    }

    public function getOptions(): Options
    {
        return $this->options;
    }

    public function captureMessage(string $message, ?Severity $level = null, ?Scope $scope = null): void
    {
        $event = Event::createEvent();
        $event->setMessage($message);
        $event->setLevel($level);

        $this->captureEvent($event, null, $scope);
    }

    public function captureException(\Throwable $exception, ?Scope $scope = null): void
    {
        $this->captureEvent(Event::createEvent(), EventHint::fromArray([
            'exception' => $exception,
        ]), $scope);
    }

    public function captureEvent(Event $event, ?EventHint $hint = null, ?Scope $scope = null): void
    {
        $event = $this->prepareEvent($event, $hint, $scope);

        if (null === $event) {
            return;
        }

        Promise\rethrow($this->transport->send($event));
    }

    public function captureLastError(?Scope $scope = null): void
    {
        $error = \error_get_last();

        if (null === $error || ! isset($error['message'][0])) {
            return;
        }

        $exception = new \ErrorException(@$error['message'], 0, @$error['type'], @$error['file'], @$error['line']);

        $this->captureException($exception, $scope);
    }

    public function getIntegration(string $className): ?IntegrationInterface
    {
        return $this->integrations[$className] ?? null;
    }

    public function flush(?int $timeout = null): Promise
    {
        return $this->transport->close($timeout);
    }

    private function prepareEvent(Event $event, ?EventHint $hint = null, ?Scope $scope = null): ?Event
    {
        if (null !== $hint) {
            if (null !== $hint->exception && empty($event->getExceptions())) {
                $this->addThrowableToEvent($event, $hint->exception);
            }

            if (null !== $hint->stacktrace && null === $event->getStacktrace()) {
                $event->setStacktrace($hint->stacktrace);
            }
        }

        $this->addMissingStacktraceToEvent($event);

        $event->setSdkIdentifier($this->sdkIdentifier);
        $event->setSdkVersion($this->sdkVersion);
        $event->setServerName($this->options->getServerName());
        $event->setRelease($this->options->getRelease());
        $event->setTags($this->options->getTags());
        $event->setEnvironment($this->options->getEnvironment());

        $sampleRate = $this->options->getSampleRate();

        if (EventType::transaction() !== $event->getType() && $sampleRate < 1 && \mt_rand(1, 100) / 100.0 > $sampleRate) {
            $this->logger->info('The event will be discarded because it has been sampled.', ['event' => $event]);

            return null;
        }

        if (null !== $scope) {
            $previousEvent = $event;
            $event = $scope->applyToEvent($event, $hint);

            if (null === $event) {
                $this->logger->info('The event will be discarded because one of the event processors returned "null".', ['event' => $previousEvent]);

                return null;
            }
        }

        $previousEvent = $event;
        $event = ($this->options->getBeforeSendCallback())($event);

        if (null === $event) {
            $this->logger->info('The event will be discarded because the "before_send" callback returned "null".', ['event' => $previousEvent]);
        }

        return $event;
    }

    private function addMissingStacktraceToEvent(Event $event): void
    {
        if (! $this->options->shouldAttachStacktrace()) {
            return;
        }

        // We should not add a stacktrace when the event already has one or contains exceptions
        if (null !== $event->getStacktrace() || ! empty($event->getExceptions())) {
            return;
        }

        $event->setStacktrace($this->stacktraceBuilder->buildFromBacktrace(
            \debug_backtrace(DEBUG_BACKTRACE_IGNORE_ARGS),
            __FILE__,
            __LINE__ - 3
        ));
    }

    private function addThrowableToEvent(Event $event, \Throwable $exception): void
    {
        if ($exception instanceof \ErrorException) {
            $event->setLevel(Severity::fromError($exception->getSeverity()));
        }

        $exceptions = [];

        do {
            $exceptions[] = new ExceptionDataBag(
                $exception,
                $this->stacktraceBuilder->buildFromException($exception),
                new ExceptionMechanism(ExceptionMechanism::TYPE_GENERIC, true)
            );
        } while ($exception = $exception->getPrevious());

        $event->setExceptions($exceptions);
    }
}

Hub.php

<?php

use Sentry\Breadcrumb;
use Sentry\Event;
use Sentry\EventHint;
use Sentry\EventId;
use Sentry\Integration\IntegrationInterface;
use Sentry\Severity;
use Sentry\State\Scope;
use Sentry\Tracing\SamplingContext;
use Sentry\Tracing\Span;
use Sentry\Tracing\Transaction;
use Sentry\Tracing\TransactionContext;

class Hub
{
    /**
     * @var Layer[] The stack of client/scope pairs
     */
    private array $stack = [];

    public function __construct(Client $client, ?Scope $scope = null)
    {
        $this->stack[] = new Layer($client, $scope ?? new Scope());
    }

    public function getClient(): Client
    {
        return $this->getStackTop()->getClient();
    }

    public function pushScope(): Scope
    {
        $clonedScope = clone $this->getScope();

        $this->stack[] = new Layer($this->getClient(), $clonedScope);

        return $clonedScope;
    }

    public function popScope(): bool
    {
        if (1 === \count($this->stack)) {
            return false;
        }

        return null !== \array_pop($this->stack);
    }

    public function withScope(callable $callback): void
    {
        $scope = $this->pushScope();

        try {
            $callback($scope);
        } finally {
            $this->popScope();
        }
    }

    public function configureScope(callable $callback): void
    {
        $callback($this->getScope());
    }

    public function captureMessage(string $message, ?Severity $level = null): void
    {
        $this->getClient()->captureMessage($message, $level, $this->getScope());
    }

    public function captureException(\Throwable $exception): void
    {
        $this->getClient()->captureException($exception, $this->getScope());
    }

    public function captureEvent(Event $event, ?EventHint $hint = null): void
    {
        $this->getClient()->captureEvent($event, $hint, $this->getScope());
    }

    public function captureLastError(): ?EventId
    {
        $this->getClient()->captureLastError($this->getScope());
    }

    public function addBreadcrumb(Breadcrumb $breadcrumb): bool
    {
        $client = $this->getClient();

        if (null === $client) {
            return false;
        }

        $options = $client->getOptions();
        $beforeBreadcrumbCallback = $options->getBeforeBreadcrumbCallback();
        $maxBreadcrumbs = $options->getMaxBreadcrumbs();

        if ($maxBreadcrumbs <= 0) {
            return false;
        }

        $breadcrumb = $beforeBreadcrumbCallback($breadcrumb);

        if (null !== $breadcrumb) {
            $this->getScope()->addBreadcrumb($breadcrumb, $maxBreadcrumbs);
        }

        return null !== $breadcrumb;
    }

    public function getIntegration(string $className): ?IntegrationInterface
    {
        $client = $this->getClient();

        if (null !== $client) {
            return $client->getIntegration($className);
        }

        return null;
    }

    public function startTransaction(TransactionContext $context): Transaction
    {
        $transaction = new Transaction($context, $this);
        $client = $this->getClient();
        $options = null !== $client ? $client->getOptions() : null;

        if (null === $options || ! $options->isTracingEnabled()) {
            $transaction->setSampled(false);

            return $transaction;
        }

        $samplingContext = SamplingContext::getDefault($context);
        $tracesSampler = $options->getTracesSampler();
        $sampleRate = null !== $tracesSampler
            ? $tracesSampler($samplingContext)
            : $this->getSampleRate($samplingContext->getParentSampled(), $options->getTracesSampleRate());

        if (! $this->isValidSampleRate($sampleRate)) {
            $transaction->setSampled(false);

            return $transaction;
        }

        if (0.0 === $sampleRate) {
            $transaction->setSampled(false);

            return $transaction;
        }

        $transaction->setSampled(\mt_rand(0, \mt_getrandmax() - 1) / \mt_getrandmax() < $sampleRate);

        if (! $transaction->getSampled()) {
            return $transaction;
        }

        $transaction->initSpanRecorder();

        return $transaction;
    }

    public function getTransaction(): ?Transaction
    {
        return $this->getScope()->getTransaction();
    }

    public function setSpan(?Span $span): Hub
    {
        $this->getScope()->setSpan($span);

        return $this;
    }

    public function getSpan(): ?Span
    {
        return $this->getScope()->getSpan();
    }

    private function getScope(): Scope
    {
        return $this->getStackTop()->getScope();
    }

    private function getStackTop(): Layer
    {
        return $this->stack[\count($this->stack) - 1];
    }

    private function getSampleRate(?bool $hasParentBeenSampled, float $fallbackSampleRate): float
    {
        if (true === $hasParentBeenSampled) {
            return 1;
        }

        if (false === $hasParentBeenSampled) {
            return 0;
        }

        return $fallbackSampleRate;
    }

    private function isValidSampleRate(float $sampleRate): bool
    {
        if ($sampleRate < 0 || $sampleRate > 1) {
            return false;
        }

        return true;
    }
}

HubFactory.php

<?php

class HubFactory
{
    private HttpClient $httpClient;
    private LoggerInterface $logger;
    private ?string $dsn;
    private string $environment;
    private string $projectDir;

    public function __construct(
        HttpClient $httpClient,
        LoggerInterface $logger,
        ?string $dsn,
        string $environment,
        string $projectDir
    ) {
        $this->httpClient = $httpClient;
        $this->logger = $logger;
        $this->dsn = $dsn;
        $this->environment = $environment;
        $this->projectDir = $projectDir;
    }

    public function create(): Hub
    {
        $options = new Options(
            [
                'dsn' => $this->dsn,
                'environment' => $this->environment,
                'default_integrations' => false,
                'in_app_exclude' => [
                    $this->projectDir,
                ],
                'in_app_include' => [
                    $this->projectDir . '/api/src',
                ],
                'max_request_body_size' => 'none',
                'send_default_pii' => true,
                'context_lines' => 10,
                'max_value_length' => 2 ** 14,
                'tags' => [
                    'php_uname' => PHP_OS,
                    'php_sapi_name' => PHP_SAPI,
                    'php_version' => PHP_VERSION,
                ],
            ]
        );
        $options->setIntegrations([new FrameContextifierIntegration()]);

        $client = new Client(
            $options,
            new Transport(
                $options,
                $this->httpClient,
                new PayloadSerializer(),
                $this->logger
            ),
            null,
            null,
            null,
            null,
            $this->logger
        );

        return new Hub($client, null);
    }
}

Layer.php

<?php

use Sentry\State\Scope;

class Layer
{
    private Client $client;

    private Scope $scope;

    public function __construct(Client $client, Scope $scope)
    {
        $this->client = $client;
        $this->scope = $scope;
    }

    public function getClient(): Client
    {
        return $this->client;
    }

    public function getScope(): Scope
    {
        return $this->scope;
    }

    public function setScope(Scope $scope): self
    {
        $this->scope = $scope;

        return $this;
    }
}

Transport.php

<?php

class Transport
{
    private Options $options;
    private HttpClient $httpClient;
    private PayloadSerializerInterface $payloadSerializer;
    private LoggerInterface $logger;
    private array $pendingRequests = [];

    public function __construct(
        Options $options,
        HttpClient $httpClient,
        PayloadSerializerInterface $payloadSerializer,
        ?LoggerInterface $logger = null
    ) {
        $this->options = $options;
        $this->httpClient = $httpClient;
        $this->payloadSerializer = $payloadSerializer;
        $this->logger = $logger ?? new NullLogger();
    }

    public function send(Event $event): Promise
    {
        $dsn = $this->options->getDsn();

        if (null === $dsn) {
            throw new \RuntimeException(\sprintf('The DSN option must be set to use the "%s" transport.', self::class));
        }

        if (EventType::transaction() === $event->getType()) {
            $request = new Request(
                $dsn->getEnvelopeApiEndpointUrl(),
                'POST',
                $this->payloadSerializer->serialize($event)
            );
            $request->addHeader('Content-Type', 'application/x-sentry-envelope');
        } else {
            $request = new Request(
                $dsn->getStoreApiEndpointUrl(),
                'POST',
                $this->payloadSerializer->serialize($event)
            );
            $request->addHeader('Content-Type', 'application/json');
        }

        return call(function () use ($event, $request): \Generator {
            try {
                $this->authenticate($request);
                $key = \array_key_last($this->pendingRequests) + 1;
                $this->pendingRequests[$key] = $request;
                /** @var \Amp\Http\Client\Response $response */
                $response = yield $this->httpClient->request($request);
            } catch (\Throwable $exception) {
                $this->logger->error(
                    \sprintf('Failed to send the event to Sentry. Reason: "%s".', $exception->getMessage()),
                    ['exception' => $exception, 'event' => $event]
                );

                return new Success();
            }

            unset($this->pendingRequests[$key]);

            if ($response->getStatus() < 200 || $response->getStatus() >= 300) {
                $msg = \sprintf(
                    'Failed to send the event to Sentry. Received status code: %d',
                    $response->getStatus()
                );

                $this->logger->error($msg);

                return new Success();
            }

            return new Success();
        });
    }

    public function close(?int $timeout = null): Promise
    {
        $promise = Promise\timeout(Promise\all($this->pendingRequests), $timeout);

        $promise->onResolve(function () {
            $this->pendingRequests = [];
        });

        return $promise;
    }

    private function authenticate(Request $request): void
    {
        $dsn = $this->options->getDsn();

        if (null === $dsn) {
            return;
        }

        $data = [
            'sentry_version' => Client::PROTOCOL_VERSION,
            'sentry_key' => $dsn->getPublicKey(),
        ];

        if (null !== $dsn->getSecretKey()) {
            $data['sentry_secret'] = $dsn->getSecretKey();
        }

        $headers = [];

        foreach ($data as $headerKey => $headerValue) {
            $headers[] = $headerKey . '=' . $headerValue;
        }

        $request->addHeader('X-Sentry-Auth', 'Sentry ' . \implode(', ', $headers));
    }
}

Next, we need to have an PSR-Logger (for amphp) and the http-client, we put have this early somewhere in our bootstrap file:

bootstrap.php

<?php

// Creating a log handler in this way allows the script to be run in a cluster or standalone.
if (Cluster::isWorker()) {
    $logHandler = Cluster::createLogHandler();
} else {
    $logHandler = new StreamHandler(ByteStream\getStdout());
    $logHandler->setFormatter(new ConsoleFormatter());
}

$logger = new Logger('worker-' . Cluster::getId());
$logger->pushHandler($logHandler);

$httpClient = HttpClientBuilder::buildDefault();

This is how to build Sentry now:

<?php

$hubFactory = new HubFactory(
    $httpClient,
    $logger,
    \getenv('SENTRY_DSN'),
    \getenv('APPLICATION_ENV'),
    __DIR__ . '/../../'
);

global $sentry;
$sentry = $hubFactory->create();

function captureException(\Throwable $e): void
{
    global $sentry;

    $sentry->captureException($e);
}

function captureExceptionWithScope(callable $callback, \Throwable $e): void
{
    global $sentry;

    $sentry->withScope(function (Scope $scope) use ($callback, $sentry, $e) {
        $callback($scope);

        $sentry->captureException($e);
    });
}

function captureExceptionFromRequest(\Throwable $e, Request $request): Promise
{
    return call(function () use ($e, $request): Generator {
        $body = yield $request->getBody()->buffer();

        captureExceptionWithScope(
            function (Scope $scope) use ($request, $body) {
                $scope->setExtra(
                    'request',
                    [
                        'uri' => $request->getUri()->getPath(),
                        'query' => $request->getUri()->getQuery(),
                        'post' => $body,
                    ]
                );

				// note that this is how I store authentication information on
                // the request, this might be different for you
                if ($request->hasAttribute('auth_info')) {
                    $info = $request->getAttribute('auth_info');

                    $scope->setUser([
                        'email' => $info['email'],
                        'name' => $info['name'],
                    ]);
                }
            },
            $e
        );
    });
}

Now within some background jobs, I can do this:

<?php

try {
    // something
} catch (Throwable $e) {
    captureException($e);
}

And to catch exceptions from web requests:

<?php

try {
    // something
} catch (Throwable $e) {
	yield captureExceptionFromRequest($e, $request);
}

That's it!

The future of prooph components

There is much development going on in the prooph team at the moment. The new event-store-client is more stable every day and development of event-store v8 is beginning soon (a lot of planning and experimenting is done already).

So naturally it was time to check if other components could need some new major version as well. There was a bit of a discussion on changes to prooph/common (#70, #71, #72) and a new prototype for snapshotter and event-sourcing component was created as well.

This was the point, where I stopped and started rethinking it completely. After I could put my thoughts together I spoke with Alexander Miertsch (the other prooph maintainer). As usual, even though we are two different people, we share the same mind and could agree pretty fast on how to proceed further.

So here's the deal...

Prooph hereby announces that the development of prooph/service-bus and prooph/event-sourcing will be dropped. This also includes all snapshot-store implementations as well as message producers.

Here is the full list of the components:

- https://github.com/prooph/service-bus
- https://github.com/prooph/event-sourcing
- https://github.com/prooph/common
- https://github.com/prooph/event-store-bus-bridge
- https://github.com/prooph/snapshotter
- https://github.com/prooph/http-middleware
- https://github.com/prooph/psb-bernard-producer
- https://github.com/prooph/pdo-snapshot-store
- https://github.com/prooph/memcached-snapshot-store
- https://github.com/prooph/mongodb-snapshot-store
- https://github.com/prooph/humus-amqp-producer
- https://github.com/prooph/annotations
- https://github.com/prooph/psb-http-producer
- https://github.com/prooph/psb-enqueue-producer
- https://github.com/prooph/snapshot-store
- https://github.com/prooph/arangodb-snapshot-store
- https://github.com/prooph/psb-zeromq-producer
- https://github.com/prooph/service-bus-zfc-rbac-bridge

All components will receive support until December 31, 2019 and will then be marked as deprecated and receive no further support from the prooph core team.

Let me explain why we decided this:

Event-Sourcing:
In fact, we recommend not using any framework or library as part of your domain model, and building the few lines of code needed to implement a left fold yourself - partly to ensure understanding and partly to keep your domain model dependency-free. The event-sourcing component was always meant to be a blueprint for a homegrown implementation but most people installed and used it as is, thus the prooph core team was complicit in advertising bad practices.

Don't worry, we will also ship some blueprints as inspiration on how to implement an aggregate root and aggregate repository, but this time, it's part of the documentation and not a repository that you could install again. You can still copy / paste from it and make it run, it's your choice.

Service-Bus:
When it was originally developed, there was no good alternative out there. In the meantime symfony shipped its own messenger component (I looked into it, it looks really great!) and sooner or later, it will probably have more downloads then our famous service-bus implementation.

This also means we now have a clear focus on developing prooph/event-store and provide even better documentation for it. The next goals are first to stabilize the new event-store-client and write documentation and more tests for it, and then start with the development of prooph/event-store v8.

Why there will be no Kafka EventStore in prooph

tl;dr

When even Greg Young, the author of the original EventStore implementation (http://geteventstore.com/) says that it's a bad idea to implement a Kafka EventStore, than it's a bad idea. The prooph-team will not provide a Kafka EventStore implementation.

Before we begin, let's see what requirements we need from an event-store:

- Concurrency checks
  When an event with same version is appended twice to the event-store, only the first attempt is allowed to succeed. This is very important, imagine you have multiple processes inserting event to an existing stream, let's say we have an an existing aggregate with only one event (version 1). Now two processes insert two events, so we have: event 1, event 2, event 2', event 3, event 3'. Next another process is inserting an event 4. If the consumer of the stream has to decide whether or not an event belongs to the stream, it's a hard decision now, because we could have the following possible event streams: event 1, event 2, event 3, event 4 or event 1, event 2', event 3, event 4 or event 1, event 2, event 3', event 4 or event 1, event 2', event 3', event 4. Additionally you can't rely on timing (let's say event 2 was inserted slightly before event 2'), because on a different data center the order could be the other way around. The matter complicates the further the event stream goes. And this is not a blockchain-like problem, where simply the longest chain wins, because sometimes, there will be no more additional events to an existing stream. I have to add, that this concurrency check requirement and version constraint might not be needed for all use-cases, in some applications it might be okay to just record whatever happened and versions / order don't matter at all (or not that much), but for a general purpose event-store implementation (where you don't wanna put dozens of warnings, and stuff), this will only bring problems and lot of bug-reports.

- One stream per aggregate
  In the original event-store implementation of Greg Young (https://github.com/EventStore/EventStore), there is by default one stream per aggregate. That means that not all events related to aggregate type "User" are stored in a single stream, but that we have one stream for each aggregate, f.e. "User-<user_id_1>", "User-<user_id_2>", "User-<user_id_3>", ...
  This option is also available for prooph-event-store, limiting the usage to disallow this strategy is possibile, but not really wanted.
  To quote Greg Young: "You need stream per aggregate not type. You can do single aggregate instance for all instances but it's yucky"

- Store forever
  While at first glance obvious, the event store should persist the event forever, it's not allowed to be removed, garbage collected or deleted on server shutdown.

- Querying event offset
  Another quite obvious thing to consider at first: Given you loaded an aggregate from a snapshot, you already have it at a specific version (let's say event version is 10 f.e.). Then you only need to load events starting from 11. This is especially important, once you have thousands of events in an aggregate (imagine you would need to load all 5000 events again, instead of only the last 3, event when you have a snapshot). Even more important when one stream per aggregate is not possible.
 

Now let's look at what Kafka has to offer:

- Concurrency checks

  Well, Kafka hasn't that. It would not be such a problem with something like actor model, to quote Greg Young again:
  >> I also really like the idea of in memory models (especially when built
  >> up as actors in say erlang or akka). One of the main benefits here is
  >> that the actor infrastructure can assure you a single instance in the
  >> cluster and gets rid of things like the need for optimistic concurrency.
  >>
  >> Greg
  This this one is a really big issue.

- One stream per aggregate
  If I have thousands of aggregates and each have a topic, Kafka (and ZooKeeper specifically) will explode. These are real problems with Kafka. ZooKeeper can't handle 10M partitions right now.

- Store forever
  On Kafka events expire! Yes really, they expire! Fortunately enough, with never versions of Kafka, you can configure it to not expire messages at all (day saved!).

- Querying event offset
  Here we go, that's not possibile with Kafka! Combine that with the "One stream per aggregate" problem, and here we go full nightmare. It's simply not reasonable to read millions or even billions of events, just to replay the 5 events you're interessted in to your aggregate root.

Way around some limitations:

- Use some actor modelish implementation (like Akka) - this would solve the "Concurrency checks" issue and well as the
  "Queryingevent offset" issue, because you can use stuff like "idempotent producer semantics" (to track producer id and position in event stream) and in-memory checks of aggregates, to go around the concurrency checks requirement. But prooph are PHP components and even if we would implement some akka-like infrastructure, that would be a rare use-case that someone wanted to use this implementation.
 
 
Another interessting quote from Greg Young about an Kafka EventStore:

>> Most systems have some amount of data that is "live" and some much
>> larger amount of data that is essentially "historical" a perfect example
>> of this might be mortgage applications in a bank. There is some tiny %
>> that are currently in process in the system and a vast number that are
>> "done".
>>
>> If you wanted to just put everything into one stream you would need to
>> hydrate all of them and keep them in memory (even the ones you really
>> don't care about any more).
>>
>> This can turn into a very expensive operation/decision.
>>
>> Cheers,
>>
>> Greg

and also this one:

>> Most of the systems discussed are not really event sourced they just
>> raise events and are distributing them. They do not keep their events as
>> their source of truth (they just throw them away). They don't do things
>> like replaying (which is kind of the benchmark)
>>
>> Not everyone who sends a message is "event sourcing"
>>
>> Greg

Final thoughts:

Kafka is great for stream processing. With enqueue (github.com/php-enqueue/enqueue-dev) and prooph's enqueue-producer (github.com/prooph/psb-enqueue-producer/) you can already send messages to Kafka for processing. So send your messages to Kafka, if you need or want to.
In my opinion a Kafka EventStore implementation would be very limited and not useful for most of the PHP applications build. Therefor I think there will never be a Kafka EventStore implementation (not in prooph, nor in any other programming language - correct me if I'm wrong and you know an open source Kafka EventStore implementation somewhere!).
When even Greg Young things a Kafka EventStore is a bad idea, I'm at least not the only one out there.

References:

Most of the Greg Young quotes about an Kafka EventStore are taken from here: https://groups.google.com/forum/#!topic/dddcqrs/rm02iCfffUY
I recommend this thread for everyone you wants to dig deaper into the problems with Kafka as an EventStore.

Authorization and Event Sourcing with prooph and ZF2 / ZF3

Authorization with prooph components and event sourced aggregates roots are a common problem to solve. Here is a short explanation on how to do this using Zend\Authentication. First of all, we need an aggregate root class for it. Here is a minimal example with some basic properties.

<?php

declare (strict_types=1);

namespace My\Identity\Model\Identity

class Identity extends \Prooph\EventSourcing\AggregateRoot
{
    /**
     * @var IdentityId
     */
    protected $identityId;

    /**
     * @var EmailAddress
     */
    protected $emailAddress;

    /**
     * @var string
     */
    protected $passwordHash;

    /**
     * @var Role[]
     */
    protected $roles;

    public function login(string $password) : bool
    {
        if (password_verify($password, $this->passwordHash)) {
            $this->recordThat(IdentityLoggedIn::with($this->identityId));

            if (password_needs_rehash($this->passwordHash, PASSWORD_DEFAULT)) {
                $this->rehashPassword($password);
            }

            return true;
        }

        $this->recordThat(IdentityLoginDenied::with($this->identityId));

        return false;
    }

    public function logout()
    {
        $this->recordThat(IdentityLoggedOut::with($this->identityId));
    }

    public function rehashPassword(string $password)
    {
        $passwordHash = password_hash($password, PASSWORD_DEFAULT);

        $this->recordThat(IdentityPasswordWasRehashed::withData(
            $this->identityId,
            $this->passwordHash,
            $passwordHash
        ));
    }

    protected function whenIdentityLoggedIn(IdentityLoggedIn $event)
    {
    }

    protected function whenIdentityLoginDenied(IdentityLoginDenied $event)
    {
    }

    protected function whenIdentityLoggedOut(IdentityLoggedOut $event)
    {
    }

    protected function whenIdentityPasswordWasRehashed(IdentityPasswordWasRehashed $event)
    {
        $this->passwordHash = $event->newPasswordHash();
    }
}

Additionally, we will need a read only version of the aggregate root:

<?php

declare (strict_types=1);

namespace My\Identity\Model\Identity\ReadOnly;

use My\Identity\Model\Identity\IdentityId;
use My\Identity\Model\Identity\Role;
use My\SharedKernel\Model\EmailAddress;
use ZfcRbac\Identity\IdentityInterface;

class Identity implements IdentityInterface
{
    /**
     * @var IdentityId
     */
    protected $identityId;

    /**
     * @var EmailAddress
     */
    protected $emailAddress;

    /**
     * @var Role[]
     */
    protected $roles;

    public function __construct(
        IdentityId $identityId,
        EmailAddress $emailAddress,
        array $roles
    ) {
        Assertion::notEmpty($roles);
        Assertion::allIsInstanceOf($roles, Role::class);

        $this->identityId = $identityId;
        $this->emailAddress = $emailAddress;
        $this->roles = $roles;
    }

    public function identityId() : IdentityId
    {
        return $this->identityId;
    }

    public function emailAddress() : EmailAddress
    {
        return $this->emailAddress;
    }

    /**
     * Get the list of roles of this identity
     *
     * @return string[]
     */
    public function getRoles()
    {
        $roles = [];

        foreach ($this->roles as $role) {
            $roles[] = $role->getName();
        }

        return $roles;
    }

    public static function fromArray(array $data) : Identity
    {
        Assertion::keyExists($data, 'identityId');
        Assertion::keyExists($data, 'emailAddress');
        Assertion::keyExists($data, 'roles');
        Assertion::isArray($data['roles']);
        Assertion::notEmpty($data['roles']);

        return new self(
            IdentityId::fromString($id),
            new EmailAddress($data['emailAddress']),
            $roles
        );
    }
}

And a projector, too:

<?php

declare (strict_types=1);

namespace My\Identity\Projection\Identity;

use Assert\Assertion;
use My\Identity\Model\Identity\Event\IdentityWasCreated;
use My\Identity\Model\Identity\Event\IdentityPasswordWasRehashed;
use Doctrine\MongoDB\Collection;
use Doctrine\MongoDB\Connection;

class IdentityProjector
{
    /**
     * @var Connection
     */
    private $connection;

    /**
     * @var string
     */
    private $dbName;

    /**
     * @param Connection $connection
     * @param string $dbName
     */
    public function __construct(Connection $connection, $dbName)
    {
        Assertion::minLength($dbName, 1);

        $this->connection = $connection;
        $this->dbName = $dbName;
    }

    public function onIdentityWasCreated(IdentityWasCreated $event)
    {
        $roles = [];
        foreach ($event->roles() as $role) {
            $roles[] = $role->getName();
        }

        $data = [
            '_id' => $event->identityId()->toString(),
            'roles' => $roles,
            'emailAddress' => $event->emailAddress()->toString(),
        ];

        $collection = $this->identityReadCollection();

        $collection->insert($data);
    }

    public function onIdentityPasswordWasRehashed(IdentityPasswordWasRehashed $event)
    {
        $this->identityReadCollection()->update(
            [
                '_id' => $event->identityId()->toString(),
            ],
            [
                '$set' => [
                    'passwordHash' => $event->newPasswordHash(),
                ],
            ]
        );
    }

    private function identityReadCollection()
    {
        return $this->connection->selectCollection($this->dbName, 'identity');
    }
}

This is a very simple example, omitting the event classes and value objects. It might be worth adding some additional methods and/ or properties, when needed. The login command simply takes the email address and password as parameters, that's simple enough for us now, so what's needed is a command handler for Login / Logout.

<?php

declare (strict_types=1);

namespace My\Identity\Model\Identity\Handler;

use My\Identity\Model\Identity\Command\Login;
use My\Identity\Model\Identity\Command\Logout;
use Zend\Authentication\Adapter\ValidatableAdapterInterface as AuthAdapter;
use Zend\Authentication\AuthenticationService;

/**
 * Class LoginLogoutHandler
 * @package My\Identity\Model\Identity\Handler
 */
final class LoginLogoutHandler
{
    /**
     * @var AuthenticationService
     */
    private $authenticationService;

    /**
     * @var AuthAdapter;
     */
    private $authAdapter;

    public function __construct(
        AuthenticationService $authenticationService,
        AuthAdapter $authAdapter
    ) {
        $this->authenticationService = $authenticationService;
        $this->authAdapter = $authAdapter;
    }

    public function handleLogin(Login $command)
    {
        $this->authenticationService->clearIdentity();

        $this->authAdapter->setIdentity($command->emailAddress()->toString());
        $this->authAdapter->setCredential($command->password()->toString());

        $auth = $this->authenticationService->authenticate($this->authAdapter);

        if (! $auth->isValid()) {
            throw new \RuntimeException('not authorized');
        }
    }

    public function handleLogout(Logout $command)
    {
        $this->authenticationService->clearIdentity();
    }
}

That should be enough for now. We also need an implementation of Zend\Authentication\Storage\StorageInterface. In this case, we use MongoDB as backend.

<?php

declare (strict_types=1);

namespace My\Identity\Infrastructure;

use Assert\Assertion;
use My\Identity\Model\Identity\ReadOnly\Identity;
use Doctrine\MongoDB\Connection;
use Zend\Authentication\Storage\StorageInterface;

final class AuthenticationStorage implements StorageInterface
{
    /**
     * @var StorageInterface
     */
    private $storage;

    /**
     * @var Connection
     */
    private $connection;

    /**
     * @var string
     */
    private $dbName;

    /**
     * @var mixed
     */
    private $resolvedIdentity;

    /**
     * AuthenticationStorage constructor.
     * @param StorageInterface $storage
     * @param Connection $connection
     * @param string $dbName
     */
    public function __construct(StorageInterface $storage, Connection $connection, $dbName)
    {
        Assertion::minLength($dbName, 1);

        $this->storage = $storage;
        $this->connection = $connection;
        $this->dbName = $dbName;
    }

    /**
     * Returns true if and only if storage is empty
     *
     * @throws \Zend\Authentication\Exception\InvalidArgumentException If it is impossible to determine whether
     * storage is empty or not
     * @return boolean
     */
    public function isEmpty()
    {
        if ($this->storage->isEmpty()) {
            return true;
        }

        $identity = $this->read();

        if ($identity === null) {
            $this->clear();

            return true;
        }

        return false;
    }

    /**
     * Returns the contents of storage
     *
     * Behavior is undefined when storage is empty.
     *
     * @throws \Zend\Authentication\Exception\InvalidArgumentException If reading contents from storage is impossible
     * @return mixed
     */
    public function read()
    {
        if (null !== $this->resolvedIdentity) {
            return $this->resolvedIdentity;
        }

        $identity = $this->connection->selectCollection($this->dbName, 'identity')->findOne([
            '_id' => $this->storage->read()
        ]);

        if (! $identity) {
            $this->resolvedIdentity = null;

            return;
        }

        $this->resolvedIdentity = Identity::fromArray($identity);

        return $this->resolvedIdentity;
    }

    /**
     * Writes $contents to storage
     *
     * @param  mixed $contents
     * @throws \Zend\Authentication\Exception\InvalidArgumentException If writing $contents to storage is impossible
     * @return void
     */
    public function write($contents)
    {
        $this->resolvedIdentity = null;
        $this->storage->write($contents);
    }

    /**
     * Clears contents from storage
     *
     * @throws \Zend\Authentication\Exception\InvalidArgumentException If clearing contents from storage is impossible
     * @return void
     */
    public function clear()
    {
        $this->resolvedIdentity = null;
        $this->storage->clear();
    }
}

Next we need an implementation of Zend\Authentication\Adapter\ValidatableAdapterInterface:

<?php

declare (strict_types=1);

namespace My\Identity\Infrastructure;

use Assert\Assertion;
use My\Identity\Model\Identity\IdentityCollection;
use My\Identity\Model\Identity\IdentityId;
use My\SharedKernel\Model\StringLiteral;
use Doctrine\MongoDB\Connection;
use MongoRegex;
use Zend\Authentication\Adapter\AbstractAdapter;
use Zend\Authentication\Result;

final class ZendMongoDbAuthAdapter extends AbstractAdapter
{
    /**
     * @var IdentityCollection
     */
    private $identityCollection;

    /**
     * @var Connection
     */
    private $connection;

    /**
     * @var string
     */
    private $dbName;

    /**
     * $authenticateResultInfo
     *
     * @var array
     */
    private $authenticateResultInfo = null;

    /**
     * ZendMongoDbAuthAdapter constructor.
     * @param IdentityCollection $identityCollection
     * @param Connection $connection
     * @param string $dbName
     */
    public function __construct(
        IdentityCollection $identityCollection,
        Connection $connection,
        $dbName
    ) {
        Assertion::minLength($dbName, 1);
        $this->identityCollection = $identityCollection;
        $this->connection = $connection;
        $this->dbName = $dbName;
    }

    /**
     * Performs an authentication attempt
     *
     * @return \Zend\Authentication\Result
     * @throws \Zend\Authentication\Adapter\Exception\ExceptionInterface If authentication cannot be performed
     */
    public function authenticate()
    {
        $this->authenticateResultInfo = [
            'code'     => Result::FAILURE,
            'identity' => $this->identity,
            'messages' => []
        ];

        $collection = $this->connection->selectCollection($this->dbName, 'identity');

        $resultIdentities = $collection->find(
            [
                'emailAddress' => new MongoRegex('/^' . $this->getIdentity() . '$/i')
            ],
            [
                '_id' => true
            ]
        )->toArray();

        if (($authResult = $this->authenticateValidateResultSet($resultIdentities)) instanceof Result) {
            return $authResult;
        }

        $identity = current($resultIdentities);

        return $this->authenticateValidateResult($identity);
    }

    /**
     * authenticateValidateResultSet() - This method attempts to make
     * certain that only one record was returned in the resultset
     *
     * @param  array $resultIdentities
     * @return bool|\Zend\Authentication\Result
     */
    private function authenticateValidateResultSet(array $resultIdentities)
    {
        if (count($resultIdentities) < 1) {
            $this->authenticateResultInfo['code']       = Result::FAILURE_IDENTITY_NOT_FOUND;
            $this->authenticateResultInfo['messages'][] = 'A record with the supplied identity could not be found.';

            return $this->authenticateCreateAuthResult();
        } elseif (count($resultIdentities) > 1) {
            $this->authenticateResultInfo['code']       = Result::FAILURE_IDENTITY_AMBIGUOUS;
            $this->authenticateResultInfo['messages'][] = 'More than one record matches the supplied identity.';

            return $this->authenticateCreateAuthResult();
        }

        return true;
    }

    /**
     * Creates a Zend\Authentication\Result object from the information that
     * has been collected during the authenticate() attempt.
     *
     * @return Result
     */
    private function authenticateCreateAuthResult()
    {
        return new Result(
            $this->authenticateResultInfo['code'],
            $this->authenticateResultInfo['identity'],
            $this->authenticateResultInfo['messages']
        );
    }

    /**
     * authenticateValidateResult() - This method attempts to validate that
     * the record in the resultset is indeed a record that matched the
     * identity provided to this adapter.
     *
     * @param  array $resultIdentity
     * @return Result
     */
    private function authenticateValidateResult($resultIdentity)
    {
        $identity = $this->identityCollection->get(IdentityId::fromString($resultIdentity['_id']));

        if (! $identity) {
            $this->authenticateResultInfo['code']       = Result::FAILURE_IDENTITY_NOT_FOUND;
            $this->authenticateResultInfo['messages'][] = 'Supplied identity not found.';

            return $this->authenticateCreateAuthResult();
        }

        if (! $identity->login(new StringLiteral($this->getCredential()))) {
            $this->authenticateResultInfo['code']       = Result::FAILURE_CREDENTIAL_INVALID;
            $this->authenticateResultInfo['messages'][] = 'Supplied credential is invalid.';

            return $this->authenticateCreateAuthResult();
        }

        $this->authenticateResultInfo['code']       = Result::SUCCESS;
        $this->authenticateResultInfo['identity']   = $identity->identityId()->toString();
        $this->authenticateResultInfo['messages'][] = 'Authentication successful.';

        return $this->authenticateCreateAuthResult();
    }
}

Now we need two little factories to create our infrastructure:

<?php

declare (strict_types=1);

namespace My\Identity\Container\Infrastructure;

use My\Identity\Infrastructure\AuthenticationStorage;
use Interop\Container\ContainerInterface;
use Zend\Authentication\Storage\Session;

final class AuthenticationStorageFactory
{
    public function __invoke(ContainerInterface $container) : AuthenticationStorage
    {
        $dbName = $container->get('Config')['projection_database'];

        return new AuthenticationStorage(
            new Session(),
            $container->get('doctrine_mongo_connection'),
            $dbName
        );
    }
}

and this one:

<?php

declare (strict_types=1);

namespace My\Identity\Container\Infrastructure;

use My\Identity\Infrastructure\AuthenticationStorage;
use Interop\Container\ContainerInterface;
use Zend\Authentication\AuthenticationService;

final class AuthenticationServiceFactory
{
    public function __invoke(ContainerInterface $container) : AuthenticationService
    {
        return new AuthenticationService($container->get(AuthenticationStorage::class));
    }
}

Last thing we need to do, is configure the service manager accordingly:

<?php
return [
    'factories' => [
        \My\Identity\Infrastructure\AuthenticationStorage::class => \My\Identity\Container\Infrastructure\AuthenticationStorageFactory::class,
        \Zend\Authentication\AuthenticationService::class => \My\Identity\Container\Infrastructure\AuthenticationServiceFactory::class,
        // for prooph's guard plugins:
        \Prooph\ServiceBus\Plugin\Guard\RouteGuard::class => \Prooph\ServiceBus\Container\Plugin\Guard\RouteGuardFactory::class,
        \Prooph\ServiceBus\Plugin\Guard\FinalizeGuard::class => \Prooph\ServiceBus\Container\Plugin\Guard\FinalizeGuardFactory::class,
        \Prooph\ServiceBus\Plugin\Guard\AuthorizationService::class => \Prooph\ServiceBusZfcRbacBridge\Container\ZfcRbacAuthorizationServiceBridgeFactory::class,
    ],
];

So when I did not forget anything, that's it! With the last 3 lines of service manager config, you can even use prooph's ServiceBus ZFC-RBAC-bridge

Flywheel Adapter for ProophEvent-Store 1.0.0 released

Prooph's Event-Store components now also has a Flywheel Adapter.

Flywheel is a serverless document database which only uses flat files on your local filesystem to store the data. All the events will be stored and loaded from a choosen directory. This is well suited when you bootstrap an application and you don't need a real database server right away. It can also be a good candidate for writing functionnal tests.

But of course you must not run it in production since it is not designed to handle a huge amount of events and doesn't manage transactions.

Shout-out to Matthieu Moquet (@MattKetmo) for providing this adapter.

Inheritance with Aggregate Roots in ProophEvent-Store

If you want to make inheritance work with aggregate roots using a common repository for all subtypes, this can be achieved very easily. You need the latest ProophEvent-Store v6.1 to do this.

An example

Consider the following use case:

<?php
abstract class User extends \Prooph\EventSourcing\AggregateRoot
{
    protected $name;

    protected $email;

    public function name()
    {
        return $this->name;
    }

    public function email()
    {
        return $this->email;
    }

    protected function whenUserWasRegisterd(UserWasRegisterd $event)
    {
        $this->name = $event->name();
        $this->email = $event->email();
    }
}

class Admin extends User
{
    public static function register($name, $email)
    {
        $self = new self();
        $self->recordThat(UserWasRegisterd::withData('admin', $name, $email);

        return $self;
    }
}

class Member extends User
{
    public static function register($name, $email)
    {
        $self = new self();
        $self->recordThat(UserWasRegisterd::withData('member', $name, $email);

        return $self;
    }
}

So in order to make this work, you need 3 small changes in your application.

Step 1: Create a UserAggregateTranslator

<?php
final class UserAggregateTranslator extends \Prooph\EventSourcing\EventStoreIntegration\AggregateTranslator
{
    /**
     * @param \Prooph\EventStore\Aggregate\AggregateType $aggregateType
     * @param \Iterator $historyEvents
     * @return object reconstructed AggregateRoot
     */
    public function reconstituteAggregateFromHistory(
        \Prooph\EventStore\Aggregate\AggregateType $aggregateType, 
        \Iterator $historyEvents
    ) {
        $aggregateRootDecorator = $this->getAggregateRootDecorator();

        $firstEvent = $historyEvents->current();
        $type = $firstEvent->type();

        if ($type === 'admin') {
            return $aggregateRootDecorator->fromHistory(Admin::class, $historyEvents);
        } elseif ($type === 'member') {
            return $aggregateRootDecorator->fromHistory(Member::class, $historyEvents);
        }
    }
}

Step 2: Change the assertion method in the EventStoreUserCollection

<?php
final class EventStoreUserCollection extends 
    \Prooph\EventStore\Aggregate\AggregateRepository
{
    public function add(User $user)
    {
        $this->addAggregateRoot($user);
    }
    public function get(UserId $userId)
    {
        return $this->getAggregateRoot($userId->toString());
    }
    protected function assertAggregateType($eventSourcedAggregateRoot)
    {
        \Assert\Assertion::isInstanceOf($eventSourcedAggregateRoot, User::class);
    }
}

Step 3: Make use of your custom AggregateTranslator

<?php
final class EventStoreUserCollectionFactory
{
    public function __invoke(ContainerInterface $container)
    {
        return new EventStoreUserCollection(
            $container->get(EventStore::class),
            AggregateType::fromAggregateRootClass(User::class),
            new UserAggregateTranslator()
        );
    }
}

If you use the provided container factory (\Prooph\EventStore\Container\Aggregate\AbstractAggregateRepositoryFactory) then you can also just change the aggregate_translator key in your config to point to the new UserAggregateTranslator and register the UserAggregateTranslator in your container.

PHP-CS-Fixer in PHPStorm with File Watcher

First of all, install php-cs-fixer:

wget http://get.sensiolabs.org/php-cs-fixer.phar -O php-cs-fixer
sudo chmod a+x php-cs-fixer
sudo mv php-cs-fixer /usr/local/bin/php-cs-fixer

Second have a project-specific php-cs config file. As an example take a look here.


In PHPStorm click "File" -> "Settings" -> "Tools" -> "File Watchers"

Fill out the provided view:

Name: PHP CS Fixer
Description: fixes php code style
Show console: Error
File type: PHP
Scope: Project Files
Program: /usr/local/bin/php-cs-fixer
Arguments: --config-file=$ProjectFileDir$/.php_cs fix $FileDir$/$FileName$ -v --diff --dry-run
Working directory: $ProjectFileDir$

It should look like this:

Hint: You can also ommit the --dry-run option, to let PHP-CS-Fixer automatically fix your code.

Adding Custom Metadata to Domain Events with Prooph Event Store

When you record events with the Prooph EventStore only the payload (the event data that was recorded by your aggregate roots) gets recorded. If you use the EventStore together with the Prooph ServiceBus and enable the TransactionManager from the EventStoreBusBridge, some additional metadata gets recorded, too. That is the "causation_id" and the "causation_name" (the command id that triggered this event and the command name).

In a real-word application you might want to record some additional metadata, like "who" issued the command, what was his IP address, which user-agent did he use, and so on. This is especially useful when you want to know, which commands "John Doe" sent to the system. The simplest way to achieve this, is to use an EventStore-Plugin. Here is an example for Zend Framework 2, using the Zend\Authentication component to get the issuer and Zend\Http\PhpEnvironment\RemoteAddress to get the client's IP address.

<?php

namespace My\App\EventStore;

use ArrayIterator;
use Iterator;
use Prooph\Common\Event\ActionEvent;
use Prooph\EventStore\EventStore;
use Prooph\EventStore\Plugin\Plugin;
use Prooph\EventStore\Stream\Stream;
use Zend\Authentication\AuthenticationServiceInterface;
use Zend\Http\PhpEnvironment\RemoteAddress;

/**
 * Class EventEnricher
 * @package My\App\EventStore
 */
final class EventEnricher implements Plugin
{
    /**
     * @var AuthenticationServiceInterface
     */
    private $authenticationService;

    /**
     * @param AuthenticationServiceInterface $authenticationService
     */
    public function __construct(AuthenticationServiceInterface $authenticationService)
    {
        $this->authenticationService = $authenticationService;
    }

    /**
     * @param EventStore $eventStore
     */
    public function setUp(EventStore $eventStore)
    {
        $eventStore->getActionEventEmitter()->attachListener('create.pre', [$this, 'onEventStoreCreateStream'], -1000);
        $eventStore->getActionEventEmitter()->attachListener('appendTo.pre', [$this, 'onEventStoreAppendToStream'], -1000);
    }

    /**
     * This method takes domain events as argument which are going to be added to the event stream and
     * adds the issued_by (user id), ip_address and user_agent as metadata to each event.
     *
     * @param Iterator $recordedEvents
     * @return Iterator
     */
    private function handleRecordedEvents(Iterator $recordedEvents)
    {
        if ($this->authenticationService->hasIdentity()) {
            $issuer = $this->authenticationService->getIdentity()->getId();
        } else {
            $issuer = 'guest';
        }

        $clientIp = $this->findClientIp();
        $userAgent = isset($_SERVER['HTTP_USER_AGENT']) ? $_SERVER['HTTP_USER_AGENT'] : '';

        $enrichedRecordedEvents = [];

        foreach ($recordedEvents as $recordedEvent) {
            $recordedEvent = $recordedEvent->withAddedMetadata('issued_by', $issuer);
            $recordedEvent = $recordedEvent->withAddedMetadata('ip_address', $clientIp);
            $recordedEvent = $recordedEvent->withAddedMetadata('user_agent', $userAgent);

            $enrichedRecordedEvents[] = $recordedEvent;
        }

        return new ArrayIterator($enrichedRecordedEvents);
    }

    /**
     * Add event metadata on event store createStream
     *
     * @param ActionEvent $createEvent
     */
    public function onEventStoreCreateStream(ActionEvent $createEvent)
    {
        $stream = $createEvent->getParam('stream');

        if (! $stream instanceof Stream) {
            return;
        }

        $streamEvents = $stream->streamEvents();
        $streamEvents = $this->handleRecordedEvents($streamEvents);

        $createEvent->setParam('stream', new Stream($stream->streamName(), $streamEvents));
    }

    /**
     * Add event metadata on event store appendToStream
     *
     * @param ActionEvent $appendToStreamEvent
     */
    public function onEventStoreAppendToStream(ActionEvent $appendToStreamEvent)
    {
        $streamEvents = $appendToStreamEvent->getParam('streamEvents');
        $streamEvents = $this->handleRecordedEvents($streamEvents);

        $appendToStreamEvent->setParam('streamEvents', $streamEvents);
    }

    /**
     * Find client ip if client was behind proxy
     *
     * @return string
     */
    private function findClientIp()
    {
        $clientIp = new RemoteAddress();
        $clientIp->setUseProxy(true);
        $ip = $clientIp->getIpAddress();

        return $ip;
    }
}

This will add "issued_by", "ip_address" and "user_agent" to the metadata of all recorded events. Now you still need to enable the plugin. If you use the provided container-factories, you simply add the plugin there:

<?php

return [
    'event_store' => [
        'plugins' => [
            \My\App\EventStore\EventEnricher::class,
        ],
    ]
];

Keep in mind, that the domain model is unaware of the domain event's metadata. Therefore if you need to know the issuer f.e. in your domain model, you should write this as a CommandBus-Plugin that manipulates the command. This way you can get the issuer from the command in your command handler within the domain. An example:

<?php

namespace My\App\EventStore;

use ArrayIterator;
use My\App\Commanding\Command;
use Iterator;
use Prooph\Common\Event\ActionEvent;
use Prooph\Common\Event\ActionEventEmitter;
use Prooph\Common\Event\ActionEventListenerAggregate;
use Prooph\Common\Event\DetachAggregateHandlers;
use Prooph\EventStore\EventStore;
use Prooph\EventStore\Stream\Stream;
use Prooph\ServiceBus\CommandBus;
use Zend\Authentication\AuthenticationServiceInterface;
use Zend\Http\PhpEnvironment\RemoteAddress;

/**
 * Class CommandAndEventEnricher
 * @package My\App\EventStore
 */
final class CommandAndEventEnricher implements ActionEventListenerAggregate
{
    use DetachAggregateHandlers;

    /**
     * @var EventStore
     */
    private $eventStore;

    /**
     * @var AuthenticationServiceInterface
     */
    private $authenticationService;

    /**
     * @param EventStore $eventStore
     * @param AuthenticationServiceInterface $authenticationService
     */
    public function __construct(EventStore $eventStore, AuthenticationServiceInterface $authenticationService)
    {
        $this->eventStore = $eventStore;
        $this->authenticationService = $authenticationService;
        $this->eventStore->getActionEventEmitter()->attachListener('create.pre', [$this, 'onEventStoreCreateStream'], -1000);
        $this->eventStore->getActionEventEmitter()->attachListener('appendTo.pre', [$this, 'onEventStoreAppendToStream'], -1000);
    }

    /**
     * @param ActionEventEmitter $dispatcher
     */
    public function attach(ActionEventEmitter $emitter)
    {
        //Attach with a low priority, so that a potential message translator has done its job already
        $this->trackHandler($emitter->attachListener(CommandBus::EVENT_INITIALIZE, [$this, 'onInitialize'], -1000));
    }

    /**
     * Add the issuer id to the command
     *
     * @param ActionEvent $actionEvent
     */
    public function onInitialize(ActionEvent $actionEvent)
    {
        $command = $actionEvent->getParam(CommandBus::EVENT_PARAM_MESSAGE);

        if (! $command instanceof Command) {
            return;
        }

        if ($this->authenticationService->hasIdentity()) {
            $issuer = $this->authenticationService->getIdentity()->getId();
        } else {
            $issuer = 'guest';
        }

        $command = $command->withIssuedBy($issuer);

        $actionEvent->setParam(CommandBus::EVENT_PARAM_MESSAGE, $command);
    }

    /**
     * This method takes domain events as argument which are going to be added to the event stream and
     * adds the issued_by (user id), ip_address and user_agent as metadata to each event.
     *
     * @param Iterator $recordedEvents
     * @return Iterator
     */
    private function handleRecordedEvents(Iterator $recordedEvents)
    {
        if ($this->authenticationService->hasIdentity()) {
            $issuer = $this->authenticationService->getIdentity()->getId();
        } else {
            $issuer = 'guest';
        }

        $clientIp = $this->findClientIp();
        $userAgent = isset($_SERVER['HTTP_USER_AGENT']) ? $_SERVER['HTTP_USER_AGENT'] : '';

        $enrichedRecordedEvents = [];

        foreach ($recordedEvents as $recordedEvent) {
            $recordedEvent = $recordedEvent->withAddedMetadata('issued_by', $issuer);
            $recordedEvent = $recordedEvent->withAddedMetadata('ip_address', $clientIp);
            $recordedEvent = $recordedEvent->withAddedMetadata('user_agent', $userAgent);

            $enrichedRecordedEvents[] = $recordedEvent;
        }

        return new ArrayIterator($enrichedRecordedEvents);
    }

    /**
     * Add event metadata on event store createStream
     *
     * @param ActionEvent $createEvent
     */
    public function onEventStoreCreateStream(ActionEvent $createEvent)
    {
        $stream = $createEvent->getParam('stream');

        if (! $stream instanceof Stream) {
            return;
        }

        $streamEvents = $stream->streamEvents();
        $streamEvents = $this->handleRecordedEvents($streamEvents);

        $createEvent->setParam('stream', new Stream($stream->streamName(), $streamEvents));
    }

    /**
     * Add event metadata on event store appendToStream
     *
     * @param ActionEvent $appendToStreamEvent
     */
    public function onEventStoreAppendToStream(ActionEvent $appendToStreamEvent)
    {
        $streamEvents = $appendToStreamEvent->getParam('streamEvents');
        $streamEvents = $this->handleRecordedEvents($streamEvents);

        $appendToStreamEvent->setParam('streamEvents', $streamEvents);
    }

    /**
     * Find client ip if client was behind proxy
     *
     * @return string
     */
    private function findClientIp()
    {
        $clientIp = new RemoteAddress();
        $clientIp->setUseProxy(true);
        $ip = $clientIp->getIpAddress();

        return $ip;
    }
}

In this case you need to add the enricher as CommandBus-Plugin instead of an EventStore-Plugin of course:

<?php

return [
    'service_bus' => [
        'command_bus' => [
            'plugins' => [
                \Prooph\EventStoreBusBridge\TransactionManager::class,
                \My\App\EventStore\CommandAndEventEnricher::class
            ],
        ],
    ],
];

You noticed perhaps that we need to use a special command class that knows the issuer. That is easy to achieve:

<?php

namespace My\App\Commanding;

use Assert\Assertion;
use Prooph\Common\Messaging\Command as ProophCommand;
use Prooph\Common\Messaging\PayloadConstructable;
use Prooph\Common\Messaging\PayloadTrait;

/**
 * Class Command
 * @package My\App\Commanding
 */
abstract class Command extends ProophCommand implements PayloadConstructable
{
    use PayloadTrait;

    /**
     * Returns a new instance of the message with given version
     *
     * @param string $issuer
     * @return Command
     */
    public function withIssuedBy($issuer)
    {
        Assertion::string($issuer);

        $messageData = $this->toArray();

        $messageData['issued_by'] = $issuer;

        return static::fromArray($messageData);
    }

    /**
     * @return string
     */
    public function issuedBy()
    {
        return $this->metadata['issued_by'];
    }
}

Now you only need to extend your commands from this base-class.

Note: If you work with Prooph EventStore, you don't need to extends the base-classes from Prooph, like Command. You're free to implement them by yourself for your needs.

Introducing the prooph-components for DDD, EventSourcing & CQRS in PHP

Prooph EventStore v6.0 beta 1 is now available with the final version coming along very soon. Time to write a short blogpost about what's new in it.

- History Replay
- Snapshot Support
- Apply Events Late
- EventStream Iterator
- interop config support
- AggregateRepository Factory

History Replay

The read model can be regenerated from history at any point in time by replaying recorded events.

Snapshot Support

If you have way to many events to replay in order to reconstruct the aggregate root, why not take a snapshot? The prooph components ship with different snapshot adapters like MongoDB, Doctrine DBAL and even Memcached backends. So replaying of thousands of events is not a burden any more.

Apply Events Late

Events are only applied after transaction commit to ensure that the aggregate root can never reach an invalid state. This is especially useful for long-running CLI scripts.

EventStream iterator

The event stream is now implemented as an Iterator instead of a simple array. This reduces the memory usage a lot when replaying big event streams.

Interop config support

The components ship with ready-to-use container-interop factories using https://github.com/sandrokeil/interop-config. This makes it really simple to configure the factories.

Want more? Visit getprooph.org to check out the documentation or try out our demo application proophessor-do. Pick up a task and at the sample application to get started with event sourcing and get your hands a little dirty.

Integretion of RabbitMQ in Zend Framework 2

When it comes to asynchronous processing in php, there is a robust way to do it, with RabbitMQ. Using RabbitMQ (an implementation of the AMQP protocal with some nice additional features) is pretty straightforward using the PHP AMQP Extension. What was missing so far was a nice Zend Framework 2 Integration, a generic consumer implementation, a simple RPC-client & -server implementation.

The producer interface is very simple:

namespace HumusAmqpModule;

interface ProducerInterface
{
    /**
     * @param string $body
     * @param string $routingKey
     * @param array|\Traversable|MessageAttributes|null $attributes
     */
    public function publish($body, $routingKey = '', $attributes = null);

    /**
     * @param array $bodies
     * @param string $routingKey
     * @param array|\Traversable|MessageAttributes|null $attributes
     */
    public function publishBatch(array $bodies, $routingKey = '', $attributes = null);
}

On the other hand we have an extendable consumer interface:

namespace HumusAmqpModule;

use AMQPChannel;
use AMQPEnvelope;
use AMQPQueue;

interface ConsumerInterface
{
    /**
     * Flag for message ack
     */
    const MSG_ACK = 1;

    /**
     * Flag for message defer
     */
    const MSG_DEFER = 0;

    /**
     * Flag for reject and drop
     */
    const MSG_REJECT = -1;

    /**
     * Flag for reject and requeue
     */
    const MSG_REJECT_REQUEUE = -2;

    /**
     * Start consumer
     *
     * @param int $msgAmount
     */
    public function consume($msgAmount = 0);

    /**
     * @return bool
     */
    public function flushDeferred();

    /**
     * @param AMQPEnvelope $message
     * @param AMQPQueue $queue
     * @return bool|null
     */
    public function handleDelivery(AMQPEnvelope $message, AMQPQueue $queue);

    /**
     * @return void
     */
    public function handleShutdownSignal();
}

The consumer supports the following:

- defer message for later decision whether to ack or nack
- ack & nack blocks of messages
- configure prefetch count
- automatic exchange and queue creation
- set callback for handling delivery of messages
- set callback for handling flush deferred messages
- set error callback
- set configurable logger
- idle timeout
- non-blocking
- configurable wait timeout
- unix signal handler support
- generic cunsumer controller and cli integration
- supervisor support
- integration in HumusSupervisorModule

Even more to come....

There is also a manual at Read the Docs and a Demo-Module incl. RPC-Client & -Server Example. Thew manual and the demo module will help you getting startet with Queuing in Zend Framework 2 without any pre-existing knowledge and guide you how to create topic-exchanges, header-exchanges, fanout-exchagesn, how to set up dead-lettering, message timeout, and so on.

Your contributions are welcome!

Lenovo Yoga 3 Pro rotate script for linux

TouchscreenDevice='ATML1000:00 03EB:8A10'
TouchpadDevice='SYNA2B22:00 06CB:2714'

These are the settings for my lenovo yoga 3 pro. If this is not working for you, find your Touchscreen and Touchpad device with xinput.

See also my gist at: https://gist.github.com/prolic/21673f0909c0cb5e2114

 

With credits to Ruben Barkow: https://gist.github.com/rubo77/daa262e0229f6e398766

#!/bin/bash
# This script rotates the screen and touchscreen input 90 degrees each time it is called, 
# also disables the touchpad, and enables the virtual keyboard accordingly

# by Ruben Barkow: https://gist.github.com/rubo77/daa262e0229f6e398766

#### configuration
# find your Touchscreen and Touchpad device with `xinput`
TouchscreenDevice='ATML1000:00 03EB:8A10'
TouchpadDevice='SYNA2B22:00 06CB:2714'

if [ "$1" = "--help"  ] || [ "$1" = "-h"  ] ; then
echo 'Usage: rotate-screen.sh [OPTION]'
echo
echo 'This script rotates the screen and touchscreen input 90 degrees each time it is called,' 
echo 'also disables the touchpad, and enables the virtual keyboard accordingly'
echo
echo Usage:
echo ' -h --help display this help'
echo ' -j (just horizontal) rotates the screen and touchscreen input only 180 degrees'
echo ' -n always rotates the screen back to normal'
exit 0
fi

touchpadEnabled=$(xinput --list-props "$TouchpadDevice" | awk '/Device Enabled/{print $NF}')
screenMatrix=$(xinput --list-props "$TouchscreenDevice" | awk '/Coordinate Transformation Matrix/{print $5$6$7$8$9$10$11$12$NF}')

# Matrix for rotation
# ⎡ 1 0 0 ⎤
# ⎜ 0 1 0 ⎥
# ⎣ 0 0 1 ⎦
normal='1 0 0 0 1 0 0 0 1'
normal_float='1.000000,0.000000,0.000000,0.000000,1.000000,0.000000,0.000000,0.000000,1.000000'

#⎡ -1  0 1 ⎤
#⎜  0 -1 1 ⎥
#⎣  0  0 1 ⎦
inverted='-1 0 1 0 -1 1 0 0 1'
inverted_float='-1.000000,0.000000,1.000000,0.000000,-1.000000,1.000000,0.000000,0.000000,1.000000'

# 90° to the left 
# ⎡ 0 -1 1 ⎤
# ⎜ 1  0 0 ⎥
# ⎣ 0  0 1 ⎦
left='0 -1 1 1 0 0 0 0 1'
left_float='0.000000,-1.000000,1.000000,1.000000,0.000000,0.000000,0.000000,0.000000,1.000000'

# 90° to the right
#⎡  0 1 0 ⎤
#⎜ -1 0 1 ⎥
#⎣  0 0 1 ⎦
right='0 1 0 -1 0 1 0 0 1'

if [ $screenMatrix == $normal_float ] && [ "$1" != "-n" ]
then
  echo "Upside down"
  xrandr -o inverted
  xinput set-prop "$TouchscreenDevice" 'Coordinate Transformation Matrix' $inverted
  xinput disable "$TouchpadDevice"
  # Remove hashtag below if you want pop-up the virtual keyboard  
  #onboard &
elif [ $screenMatrix == $inverted_float ] && [ "$1" != "-j" ] && [ "$1" != "-n" ]
then
  echo "90° to the left"
  xrandr -o left
  xinput set-prop "$TouchscreenDevice" 'Coordinate Transformation Matrix' $left
  xinput disable "$TouchpadDevice"
  #killall onboard
elif [ $screenMatrix == $left_float ] && [ "$1" != "-j" ] && [ "$1" != "-n" ]
then
  echo "90° to the right"
  xrandr -o right
  xinput set-prop "$TouchscreenDevice" 'Coordinate Transformation Matrix' $right
  xinput disable "$TouchpadDevice"
  #killall onboard
else
  echo "Back to normal"
  xrandr -o normal
  xinput set-prop "$TouchscreenDevice" 'Coordinate Transformation Matrix' $normal
  xinput enable "$TouchpadDevice"
  #killall onboard
fi

Minifesto - MANIFESTO FOR MINIMALIST SOFTWARE ENGINEERS

• Fight for Pareto's law, look for the 20% of effort that will give you the 80% of results.
• Prioritize, minimalism isn't about not doing things but about focusing first in the important.
• Perfect is enemy of good, first do it, then do it right, then do it better.
• Kill the baby, don't be afraid of starting all over again. Fail soon, learn fast.
• Add value. Think constantly how you can help your team and position yourself in that field/skill.
• Basics, first. Follow always a top-down thinking starting by the best-practises of CS.
• Think different. Simple is harder than complex, which means you'll need to use your creativity.
• Synthesis is the key of communication. We have to write code for humans not machines.
• Keep it plain. Try to keep your designs with few layers of indirection.
• Clean kipple and redundancy. Minimalism is all about removing distractions.

more: http://www.minifesto.org/

I am currently working on the german translation and will get the russian translation from a friend, too.

Sending stream responses with Zend Framework 2, Part 1

Sending stream responses is a new feature in Zend Framework 2.1. This short introduction will help you getting startet on how to send stream responses in Zend Framework 2.
Most times an application would deliver file downloads or video streaming with static assets, delivered directly frm Apache or Nginx. But sometimes there are restrictions: Only logged-in users should be able to download this file, watch this content, perhabs even a more complicated permission system. Then the application needs to do some permission checks and stream the response.
In this example we assume a simple file download, we deliver the file as stream response, because this is much less memory consuming and more efficient, compared to grabbing the file content and pushing it to the response body of the default response object.

namespace Application\Controller;

use Zend\Http\Headers;
use Zend\Http\Response\Stream;
use Zend\Mvc\Controller\AbstractActionController;

class IndexController extends AbstractActionController
{
    public function fileAction()
    {
        $file = '/path/to/my/file.txt';
        
        $response = new Stream();
        $response->setStream(fopen($file, 'r'));
        $response->setStatusCode(200);
        $response->setStreamName(basename($file));

        $headers = new Headers();
        $headers->addHeaders(array(
            'Content-Disposition' => 'attachment; filename="' . basename($file) .'"',
            'Content-Type' => 'application/octet-stream',
            'Content-Length' => filesize($file)
        ));
        $response->setHeaders($headers);
        return $response;
    }
}

Even more complex use cases with download resume and speed limits are also possible in the future, but this is part of my next blog post about this topic. However you can see how easy it is to send stream responses with Zend Framework 2. As we return the response object directly in the controller, no view is created and rendered, too, so this is really a very fast operation.

List of current php version in major linux distros

I tried to make a list of current php versions in major linux distributions. Please feel free to correct or add additional one per comment.

Distribution	PHP Version
Ubuntu 16.04 Xenial	7.0.4
Ubuntu 15.10 Willy	5.6.11
Ubuntu 14.04 LTS Trusty	5.5.9
Ubuntu 12.04 LTS Precise	5.3.10
Ubuntu 12.10 Quantal Debian 8 Jessie	5.4.6 5.6.14
Debian 7.5 Wheezy	5.4.4
Debian 6.0.6 Squeeze	5.3.3
Cent OS 6.3	5.3.3
Arch Linux	5.4.11
Fedora 18	5.4.9
Fedora 17	5.4.1
Mandriva 2011	5.3.20
Mandriva 2010	5.3.2
Mageia 2	5.3.19
OpenSUSE 12.2	5.3.15
OpenSUSE 13.1	5.4.20
OpenSUSE 13.2	5.6.1
Slackware 14	5.4.7
PCLinuxOS	5.3.8
Red Hat (Enterprise) Linux 6.3	5.3.3
Red Hat (Enterprise) Linux 7.0 RC	5.4.16
SUSE Linux Enterprise System 11 SP1	5.2.6
SUSE Linux Enterprise System 11 SP2	5.2.14
SUSE Linux Enterprise System 11 SP3	5.3.17

Using git-svn for projects with many branches

About branches: Git-svn would normally clone all revisions from all branches by default. This is a time-consuming process, so I strongly recommend adding only those branches, that you need for your work.
We don't specify a branches section in our git configuration, instead we are attaching branches as new remotes. That also means, that we can't create or delete branches on a svn server, they have to be created with the corresponding svn commands.

svn cp /trunk /branches/
svn rm /branches/

1) Cloning the repository, checking out trunk

First create a directory where you want your git repository to be on your harddrive, than initialize a new git repository.

mkdir gitsvn
cd gitsvn
git init

Next you have to edit your git config and add the trunk as svn-remote.

edit .git/config and add:

[svn-remote "svn"]
        url = http:///svn/trunk
        fetch = :refs/remotes/git-svn

Next you clone the repository. This will check all SVN revisions and start a download for every revision of every file of the trunk, so this will take a while.

git svn fetch

You end up with a local branch "master" that tracks the svn remote "trunk". Type "git svn info" to show information.

git svn info

2) Keeping things up to date

You should never work on the master branch, that one is only reflecting the trunk, it's a mirror, so please never change files here. Just update the master to reflect the trunk, that's all.

git svn fetch
git svn merge trunk –ff

The --ff flag forces a fast-forward.

3) Creating new local branches and rebase

Sometimes, you just want to create a new local branch and work on it. (Perhabs a small bugfix branch, that immediatly gets merged into trunk.)

git checkout -b bugfixbranch trunk

... do some work …

git commit -m "nice explaining commit message"

All unit tests are good, and you want to push that branch perhabs directory to svn trunk. But first, we need to rebase. Someone else could have made commits to trunk in the meantime, so you have to apply his changes first, than yours, that means rewriting your local history.

git svn fetch
git rebase -i trunk

.. now you have the choice, you can pick, reword, edit, squash and fix commits …

git svn dcommit –dry-run

.. it's always recommended to make a dry-run first, so you don't push to the wrong location accidentialy

git svn dcommit

4) Adding a new svn remote branch

Because we have so many branches, that git would force you to download them all, we just specify the explicit branches we want to track.

git config --add svn-remote.branchname.url http:///svn/branchname
git config --add svn-remote.branchname.fetch :refs/remotes/branchname
git svn fetch branchname [-r]
git checkout -b localbranchname -t branchname
git svn rebase branchname

I recommend that you specify the revision number here, otherwise git will download every revision, starting with 0 first. If your branch was created at revision 100000, you will have to wait a long time.

git checkout -b localbranchname branchname
git svn info

5) Reintegrating a single commit into trunk

In the most easiest case, you have a branch with just one single commit, so you can just cherry-pick it.

git log -2

Output:

commit 2db1182ddc0e85b3a9c468bd7610a58c3bb99d0a
Author: sascha.prolic 
Date:  Tue Jan 8 11:37:29 2013 +0000
    Nice explaining commit message

git-svn-id: http:///svn/branches/branchname@100000 d12c8f96-4347-4129-8f7d-0f3f6588512b

commit c56d7da69e96a662aa9aad8cd394a381bad64807
Author: sascha.prolic 
Date:  Tue Jan 8 16:08:26 2013 +0000
    created branch

git-svn-id: http:///svn/branches/branchname@100000 d12c8f96-4347-4129-8f7d-0f3f6588512b

As you can see, the first commit is "c56d7da69e96a662aa9aad8cd394a381bad64807", but that one is just the svn copy, we don't want to merge this, so only "2db1182ddc0e85b3a9c468bd7610a58c3bb99d0a" is interessting. Lets change to master, update, run the unit tests and commit.

git checkout master
git svn fetch
git merge trunk –ff
git cherry-pick 2db1182ddc0e85b3a9c468bd7610a58c3bb99d0a

... run unit tests

git svn dcommit

6) Reintegrate a branch into trunk

git checkout -b mergebranch branchname
git svn fetch
git rebase -i trunk

Your editor should pop up with something like this in it:

pick 0c70bbe created new branch
pick a5bf234 branchname foo
pick d24360d branchname bar

Note, that we don't want to merge "0c70bbe", because that would was the svn cp command. Just delete this line. Save the file, and exit. You can even squash commit or edit commit messages. Run the unit tests again.

git checkout master
git merge trunk –ff
git merge mergebranch
git svn dcommit

7) Delete the branch checkout

The following command sequence will delete (locally only) the SVN branch and its history:

git branch -D local-branch
git branch -D -r branch
rm -rf .git/svn/branch
git gc

To remove the branch on the svn server, remove it with svn rm <branchurl>.

8) Converting a local branch into a remote branch

Basically all it amounts to is a rebase, with some svn trickery.

svn cp /trunk /branches/final-remote-name

Next you add that branch to your git config.

git checkout local-feature-branch   # This is the local branch where your prototype feature is committed.
git checkout -b final-local-name  # This makes the local branch derived from your local feature branch that will eventually become the branch that tracks the remote svn branch.
git svn info  # This will say you are following trunk.
git rebase remotes/final-remote-name

Verify

git log
git svn info

How to test a Zend Framework 2 Application

A Zend Framework 2 application has a couple of test suites. First of all, it contains Zend Framework 2 including its test suite. Then you have also your own module (or modules) which can contain its own test suites. Last but not least, you could have additional thirdparty modules (f.e. DoctrineORMModule or DoctrineModule) and thirdparty libraries (f.e. Symfony\Console, Doctrine\ORM, Doctrine\Common), which contain its own test suites.
To run all test suites for your application, you need to install phpunit and it's dependencies, then you need to run something like this:

phpunit -c vendor/zendframework/zendframework/tests/phpunit.xml
phpunit -c vendor/doctrine/common/phpunit.xml
phpunit -c modules/Application/tests/phpunit.xml
...

Well, of course you can create a simple bash script, that does the job for you. But if you want to exclude disable modules, that won't work. You have to change the script each time, you disable or install a module.

The solution: Humus PHPUnit Module

Humus PHPUnit Module is a Module for Zend Framework 2 for unit testing. It is able to test all your zf2 modules and libraries at once.

The installation is quite easy:


And you DON'T have to have phpunit installed at all! HumusPHPUnitModule will install EHER/PHPUnit through composer and you have phpunit installed in your vendor directory.

Usage is easy, too:

./vendor/bin/humusphpunit

How it works:
You have two possibilities, to enable a test suite for the humus phpunit module.
1) Using HumusPHPUnitModule\ModuleManager\Feature\PHPUnitProviderInterface

<?php

namespace MyModule;

use HumusPHPUnitModule\ModuleManager\Feature\PHPUnitProviderInterface;

class Module implements PHPUnitProviderInterface
{
    public function getPHPUnitXmlPaths()
    {
        return array(
            dirname(dirname(__DIR__)) . '/tests/phpunit.xml'
        );
    }
}

You can implement the PHPUnitProviderInterface in your own modules and provide an array of phpunit.xml files. The PHPUnit Runner collects them and runs all of them. To run the tests for thirdparty modules, that are independent from HumusPHPUnitModule, you can either write your own module, that returns phpunit.xml files for every thirdparty module you have (this can be one module per thirdparty-lib, but also a single one returning an array including all phpunit.xml files for all your thirdparty modules. Another way is using the configuration file.

2) Using configuration

<?php
return array(
    'humus_phpunit_module' => array(
        'phpunit_runner' => array(
            'Doctrine\Common' => array(
                'vendor/doctrine/common/phpunit.xml.dist'
            )
        )
    )
);

Just put this in your application configuration and run the bin script (./vendor/bin/humusphpunit).
That's all, there is also a HumusPHPUnitSampleApp - This is a demo application to show the features of Humus PHPUnit Module. It tests a sample application module with the PHPUnitListener and the Doctrine\Common lib and Zend Framework 2 with configuration.

When migration to Zend Framework 2 seems impossible

The problem

When you got a really big codebase and lots of developers working on it, releasing new features and bugfixes every 2-3 weeks, it seems impossible to migrate to Zend Framework 2, soon.
That's because you can't stop further development, you have customers waiting for the new features, they wait for bugfix releases, to get stuff running, that's buggy, you have more than 1mio lines of code and possibly > 100 active developers on that codebase. How can you migrate? I stombled accross this problem already when the first ZF2 betas came out and startet to write some code that should solve the problem.
The result is HumusMvc, it integrates Zend Framework 2's ModuleManager and ServiceManager in a ZF1 application.

The migration path

First of all, you create a new skeleton for you application. Simply clone the HumusMvcSkeletonApplication and use it as base for you skeleton app.
At the next step, you create a repository per module you have in your old ZF1 application. Put the source code in the "src/" directory of that module, create a Module.php file, name the Module, put the module configuration in it, etc. - in other words, create a simple ZF2 module out of your ZF1 application module. If you use HumusMvcAssetManager, too, you can also provide a public directory in your module and use the asset manger. That way you can override assets in a very similar way you override view scripts in a ZF1 application.

The module now can look something like this:
- MyAppDefaultModule/
    - config/
        - module.config.php
    - layouts/
    - public/
        - css/
        - js/
        - img/
    - src/
        - controllers/
        - models/
        - views/
    - tests/
    - autoload_classmap.php
    - composer.json
    - Module.php

If you use ZF1 with the Resource-Autoloader, than you don't have psr-0 compliant classnames, so define your autoloading simply with classmap autoloader.
In order to install your module, just require it with composer.

In your module.config.php you provide your front_controller configuration, you have to tell the front controller, that you have a module, where it can dispatch.

<?php
return array(
    'front_controller' => array(
        'controller_directory' => array(
            'default' => __DIR__ . '/../src/controllers'
        ),
    )
);

You can now define some resources with ZF2's service manager in your module config file. You can remove those resources from bootstrapping later. HumusMvc ships with Navigation, Translator and Locale factories, so you don't need to handle these, just configure them.
If you have something like a db connection, that was bootstrapped in your old application, you need to do this again. Therefore you use the "onBootstrap" event. Well, you could define everything right now with the service manager, but that is exactly that much work, we wanted not to do now, because that would stop the further development for quite a time.

MyAppDefaultModule\Module.php

    /**
     * Listen to the bootstrap event
     *
     * @param EventInterface $e
     * @return array
     */
    public function onBootstrap(EventInterface $e)
    {
        $application = $e->getParam('application');
        $serviceManager = $application->getServiceManager();

        $config = $serviceManager->get('Config');

        // @todo refactor later and use cleanly service manager
        $this->setupDbConnection($serviceManager, $config);

        // we need to put config in zend_registry
        // @todo refactor later
        \Zend_Registry::set('config', new \Zend_Config($config));
    }

    protected function setupDbConnection($serviceManager, $config)
    {
        if (!isset($config['myapp']['db'])) {
            throw new \RuntimeException(
                'No multi db config found.'
            );
        }
        $options = $config['myapp']['db'];

        if (isset($options['defaultMetadataCache'])) {
            $this->setDefaultMetadataCache($options['defaultMetadataCache'], $serviceManager);
            unset($options['defaultMetadataCache']);
        }

        $adapter = $options['adapter'];
        $default = $options['default'];
        unset(
            $options['adapter'],
            $options['default']
        );

        $adapter = \Zend_Db::factory($adapter, $options);

        if ($default) {
            \Zend_Db_Table::setDefaultAdapter($adapter);
        }
    }

Here is a short example you to bootstrap db connection on every request and put the application configuration into Zend_Registry.

Further refactorings

At first, I recommend to remove all "onBootstrap" methods when neccessary and put this stuff in your service manager, so you only instantiate the objects, when they are need, not on every request.
In the next refactoring rounds, you can refactor only one module at a time, so development in the other modules can still be done. Put all stuff in the service manager and use it as a service locator inside your application. Remove Zend_Registry everywhere. When you are done with that, use a php namespacer tool (or do it by manually) and refactor to a namespaced codebase. Remember, code is per module, so you can do this step module per module. Further bugfixing and development of new features is still possible.

Final migration to Zend Framework 2

You should now have a very flexible and modern Zend Framework 1 application, that makes use of a lot of stuff from Zend Framework 2, already: Service Manager, Module Manager, Event Manager, just to name a few. You don't have any expensive upfront bootstrapping any more, you moved everything to the service manger. You should have a namespaced codebase. You use ZF 2 classes much more, than ZF1 classes. - Now it's time to do the final migration.

Refactor all controllers, refactor route configuration refactor all view scripts, and so on. Now it's time for the hard work, we skipped before - but we had a couple of month time to bring our codebase to something, that _can_ be refactored to a ZF2 codebase very fast.