tag:blogger.com,1999:blog-5683149866285914564.post7883284787651195535..comments2024-03-26T23:19:10.383+08:00Comments on sasaprolic: Why there will be no Kafka EventStore in proophsasaprolichttp://www.blogger.com/profile/00508992088249141989noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-5683149866285914564.post-43361154087318687152024-03-26T17:20:03.447+08:002024-03-26T17:20:03.447+08:00That was my first article on this topic so I guess...That was my first article on this topic so I guess it will take me time to absorb the info. Was reading an article on <a href="https://www.emeriosoft.ae/digital-marketing-agency-dubai/" rel="nofollow">digital marketing services in UAE</a> before this one and it was great too.sulemanbinsheikhhttps://www.blogger.com/profile/17882176579149261191noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-33662437568086481792023-09-30T05:41:12.317+08:002023-09-30T05:41:12.317+08:00Kafka EventStore won't integrate with Prooph p...Kafka EventStore won't integrate with Prooph primarily because of architectural differences. Prooph emphasizes Event Sourcing with its own event store solutions. This separation maintains system integrity. For <a href="https://www.assignmentuk.co.uk/" rel="nofollow">assignment help UK</a>, this ensures robustness and scalability, enhancing the platform's ability to support academic needs efficiently.Marsha Kaplanhttps://www.blogger.com/profile/04386490823616657819noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-32938965659411167482023-07-15T00:03:25.124+08:002023-07-15T00:03:25.124+08:00As a design subscription service, I find it intere...As a <a href="https://11thagency.com/unlimited-graphic-design-services/" rel="nofollow">design subscription service</a>, I find it interesting to explore the reasons behind the absence of a Kafka EventStore in prooph. While Kafka is a powerful event streaming platform, the decision to not include it in prooph may be based on various factors. It could be a matter of aligning with the specific needs and goals of prooph, focusing on alternative event storage solutions, or considering the trade-offs in terms of complexity, maintainability, and scalability. Ultimately, understanding the reasoning behind such decisions helps us appreciate the careful considerations that go into shaping software frameworks and tools.Robert Leehttps://www.blogger.com/profile/16924714887510437120noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-9317751153943480392023-07-06T21:17:09.157+08:002023-07-06T21:17:09.157+08:00The absence of a Kafka EventStore in prooph may st...The absence of a Kafka EventStore in prooph may stem from various considerations, such as architectural decisions, project goals, or technical constraints. It's intriguing to explore the reasoning behind this choice, as it sheds light on the design philosophy and trade-offs made. If you're looking for expert assistance with your journal article, our reliable journal article writing services are here to support you. Our <a href="https://thesiswritinghelp.com.pk/publication/research-article-publication/" rel="nofollow">journal article writing services</a> will ensure your research is presented effectively, helping you communicate your findings with clarity and impact. Trust us to elevate your scholarly contributions.<br /><br /><br /><br /><br />keven johnhttps://www.blogger.com/profile/11428952529177765789noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-58330429198228654802023-06-14T17:24:51.852+08:002023-06-14T17:24:51.852+08:00Hello Guys! hope you all are well, The best movie ...Hello Guys! hope you all are well, The best movie I have ever seen is Inception(https://www.thestreambible.com/netflix/watch-inception-on-netflix/) this is one of the best movie that i have ever watchedJhon stevehttps://www.blogger.com/profile/03812410319752468576noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-55260954185982888312022-12-16T13:41:29.984+08:002022-12-16T13:41:29.984+08:00Hey there movie fans out there, I have brought The...Hey there movie fans out there, I have brought <a href="https://www.watchinuk.co.uk/watch-the-fallout-movie-in-uk/" rel="nofollow">The Fallout’ movie in UK</a> for you to watch online for free at Watch in Uk, the best streaming platform. Oliver Bethhttps://www.blogger.com/profile/08408985499939928418noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-27844307547992705412019-01-16T16:04:02.498+08:002019-01-16T16:04:02.498+08:00Regarding concurrency issue, I've just had a b...Regarding concurrency issue, I've just had a breakthrough while browsing this example project: https://github.com/amitayh/event-sourcing-kafka-streams.<br /><br />Basically, by publishing commands on a topic and using aggregate id as the message keys, the owner of the project was able to ensure that no command would be simultaneously handled for a given aggregate id.<br /><br />Thus, completely removing the need for concurrency check when appending the new events.<br /><br />I believe the magic happens here: https://github.com/amitayh/event-sourcing-kafka-streams/blob/master/commandhandler/src/main/scala/org/amitayh/invoices/commandhandler/CommandToResultTransformer.scala#L27-L39.<br /><br />WDYT?<br /><br />GildasGildashttps://www.blogger.com/profile/12999035916658511991noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-78470161518034986692019-01-16T15:30:27.532+08:002019-01-16T15:30:27.532+08:00Hello Ben,
I'm currently investigating on usi...Hello Ben,<br /><br />I'm currently investigating on using Kafka as an Event Store, and I have a few questions regarding your previous comment (sorry for digging up the thread).<br /><br />- it encourages you to process events directly which takes you down a more event-driven route.<br /><br />=> That is not directly related to using Kafka, but is always true when going in the Event Sourcing path, isn't it?<br /><br />- It separates the log of events (the source of truth) from the view(s) which makes it work well in multi tenant systems like microservices. This is useful because the source of truth remains (and is authoritative), but the view(s) typically change frequently.<br />- This also means microservices can share a single source of truth, but each microservice has views that are lightweight and targeted to the job that microservice needs to do, as well as being wholly owned and operated by that service.<br /><br />=> That also will be achieved when using CQRS/Event Sourcing with any event store infrastructure, isn't it?<br /><br />- You can leverage all the tools that come with a stream processing engine, right on your log of events.<br /><br />Are you talking about kafka-streams, KStream and KTable for instance?<br /><br />More generally, my main concern about using Kafka as an event store is the difficulty/impossibility to retrieve a stream of events for a given aggregate type and id. A workaround to this is to have a topic per aggregate type and consumers that index events by aggregate id. Then, when a process needs to fetch a stream of events to rebuild an aggregate (during an update operation for instance), it can rely on this separate index. Which mean that the write processes rely on a view (these indexes are projections/views of the source of truth), and not on the source of truth, which makes me very uncomfortable (as these indexes are eventual consistent by nature)!<br /><br />My second concern, as Sascha has stated, is the inability to handle concurrency check when publishing events (aka optimistic locking) with Kafka. That means that domain invariant can be silently broken and that does not sound good at all... However, adding such feature to Kafka is under discussion here: https://issues.apache.org/jira/browse/KAFKA-2260.<br /><br />Please note that these concerns could arise because I have a biased idea of what, or how, should an event sourced system work (mainly by experiencing on Prooph/Event Store), so feel free to correct me if I'm raising incorrect issues (or non issues)!<br /><br />Thanks!<br /><br />GildasGildashttps://www.blogger.com/profile/12999035916658511991noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-74127117353341146982018-09-17T14:57:13.670+08:002018-09-17T14:57:13.670+08:00RDBMS is good choice for implementing event store ...RDBMS is good choice for implementing event store but, they lack on rebuilding read models and projections, for example we have 1M stream (aka table in RDBMS on per-aggregate-id per-aggregate-type strategies),how i can project from multiple tables? <br /> Anonymoushttps://www.blogger.com/profile/08222080609309917809noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-90850759688046527522018-04-21T16:54:01.361+08:002018-04-21T16:54:01.361+08:00Hi Ben, thanks for your input. At the open source ...Hi Ben, thanks for your input. At the open source project prooph (http://getprooph.org/) we got a lot of requests for an event store implemented with Kafka. So I looked into it and this blog post is the result of my research.<br /><br />I am not saying that Kafka is a bad tool, or it cannot be used in event sourced systems at all. All I am saying is that Kafka is not suitable for an event store. By event store I mean a database that is able to store events (like http://eventstore.org/ or as in prooph we use MySQL / Postgres). This works really well and Kafka can be a nice addition for event-processing, view generation, etc. But as you said yourself, the Kafka broker is a log. I would not use the word "event store" when talking about Kafka, because an event store has already a specific meaning. People get confused quickly, if you say things like "Kafka is an event store that..." - Kafka is not an event store. As I laid out in this blog post, it's not suitable for an event store at all. But Kafka is still a great tool that can give awesome results when used in combination, no doubts about that.sasaprolichttps://www.blogger.com/profile/00508992088249141989noreply@blogger.comtag:blogger.com,1999:blog-5683149866285914564.post-84848419833157720132018-04-17T02:01:10.033+08:002018-04-17T02:01:10.033+08:00Hi there Sascha
The Kafka broker is really a log ...Hi there Sascha<br /><br />The Kafka broker is really a log so it’s best not to think of it as a database. It provides a powerful event storage layer. Atop that you add a view (i.e. the Query side of CQRS). People do this in two ways: either via the Kafka Streams API (which is part of Kafka) which provides an internal ‘state store’ where you can build views that allow you to query aggregates directly within your app. Alternatively via a database (which it can be connected to via one of the connectors).<br /><br />The approach is little different to the way people use Event Store, so I can see where the confusion would come from, but it has some nice advantages: <br />- it encourages you to process events directly which takes you down a more event-driven route. <br />- It separates the log of events (the source of truth) from the view(s) which makes it work well in multi tenant systems like microservices. This is useful because the source of truth remains (and is authoritative), but the view(s) typically change frequently. <br />- This also means microservices can share a single source of truth, but each microservice has views that are lightweight and targeted to the job that microservice needs to do, as well as being wholly owned and operated by that service. <br />- You can leverage all the tools that come with a stream processing engine, right on your log of events. <br /><br />So I see Event Store as an event sourcing database, and it definitely has its place. Kafka is more like an event store that facilitates a *set* of event driven applications through the event sourcing / command sourcing, EDA and CQRS patterns, as well as providing a lot of scalability and resiliency primitives. That’s why it tends to be used a lot with microservices. <br /><br />There is an example of a small application here: https://www.confluent.io/blog/building-a-microservices-ecosystem-with-kafka-streams-and-ksql/<br /><br />There is also a bit more detail in this free book: https://www.confluent.io/designing-event-driven-systems<br /><br />Oh, and you can store data in kafka for as long as you want (set retention.ms=-1). In fact topics with hundreds of TBs are not uncommon. <br /><br />BenBenShttps://www.blogger.com/profile/08663381616163819530noreply@blogger.com