Hello!
Since there are talks about supporting SQL Server, I am wondering if as of now Marten has a pluggable system enabling users to add and register support for additional databases?
My idea is to add a provider for the CouchDb database. I feel that since it is a "native" json-based document db, adding support for this shouldn't be too complicated if a pluggable system is already in place.
Now in terms of why: CouchDB is great for document storage (absolutely no migrations scripts are ever needed, documents are stored in json, a real-time continuous change feed available over http for every database, great clustering and multi-master scaleout story etc....) But has zero SQL support so is very limited in terms of aggregation queries. It also lacks the event sourcing tooling available in Marten.
Marten on the other hand works natively with postgres, which enables extremely flexible ways to query data. In a nutshell, my thinking is to use Marten as a sort of Mediator to facilitate a CQRS implementation, so that most writes (namely events) would be routed to CouchDb, while some, most or all model snapshots would be stored in postgresql to ease querying. The goal would basically be to use couchdb for the write side and some or most of the read side, while using postgresql for some or most of the read side.
By default I would configure most snapshots to be stored in CouchDb, while some snapshots would be stored in postgresql where advanced aggregation queries are needed whenever the need arises (which event sourcing enables to do quite easily). This would be a best of both worlds kind of thing.
Another benefit of this approach is that it could potentially also enable MartenDb users to do major version postgresql upgrades more easily and safely at scale as they would only amount to rebuilding model snapshots on a new postgresql instance running the new version. A zero downtime switchover could even be achievable relatively easily thanks to the snapshot daemon that would continually keep the new instance in sync with the latest events ingested by couchdb.
Having worked a lot with NoSQL databases and CouchDb in particular, while the development-time story of marten is great, migrations are still needed when deploying to production and it makes putting together a continuous deployment pipeline more challenging (this is the friendly feedback of a newcomer).
I feel that that Marten may be lacking something equivalent to EntityFramework's modelbuilder class whose key benefit is that it allows developers to have a representation of the state of the schema of the production database without needing to have rights to connect directly to the production database to make diffs against it. This makes letting the application automatically apply migrations on startup way less risky and way more acceptable, while keeping the deployment pipeline simple. I've been scratching my head a lot on Marten's migrations management workflow for the past few weeks but haven't been able to figure out to have something similar to what EF's modelbuilder provides (an other benefit being that it is easy to store and diff in source control). So deploying my application is quite a pain.
Right now, since I was big on using EF, then big on using CouchDb, since using marten this is the first time I need to figure out a deployment process to manually check and apply sql migrations, and I feel this is a huge step backward in my productivity. So in addition to performance and scalability, an additional goal pursued by the couchdb integration would be to keep the number of schema changes on the postgresql side very small.
Hello!
Since there are talks about supporting SQL Server, I am wondering if as of now Marten has a pluggable system enabling users to add and register support for additional databases?
My idea is to add a provider for the CouchDb database. I feel that since it is a "native" json-based document db, adding support for this shouldn't be too complicated if a pluggable system is already in place.
Now in terms of why: CouchDB is great for document storage (absolutely no migrations scripts are ever needed, documents are stored in json, a real-time continuous change feed available over http for every database, great clustering and multi-master scaleout story etc....) But has zero SQL support so is very limited in terms of aggregation queries. It also lacks the event sourcing tooling available in Marten.
Marten on the other hand works natively with postgres, which enables extremely flexible ways to query data. In a nutshell, my thinking is to use Marten as a sort of Mediator to facilitate a CQRS implementation, so that most writes (namely events) would be routed to CouchDb, while some, most or all model snapshots would be stored in postgresql to ease querying. The goal would basically be to use couchdb for the write side and some or most of the read side, while using postgresql for some or most of the read side.
By default I would configure most snapshots to be stored in CouchDb, while some snapshots would be stored in postgresql where advanced aggregation queries are needed whenever the need arises (which event sourcing enables to do quite easily). This would be a best of both worlds kind of thing.
Another benefit of this approach is that it could potentially also enable MartenDb users to do major version postgresql upgrades more easily and safely at scale as they would only amount to rebuilding model snapshots on a new postgresql instance running the new version. A zero downtime switchover could even be achievable relatively easily thanks to the snapshot daemon that would continually keep the new instance in sync with the latest events ingested by couchdb.
Having worked a lot with NoSQL databases and CouchDb in particular, while the development-time story of marten is great, migrations are still needed when deploying to production and it makes putting together a continuous deployment pipeline more challenging (this is the friendly feedback of a newcomer).
I feel that that Marten may be lacking something equivalent to EntityFramework's modelbuilder class whose key benefit is that it allows developers to have a representation of the state of the schema of the production database without needing to have rights to connect directly to the production database to make diffs against it. This makes letting the application automatically apply migrations on startup way less risky and way more acceptable, while keeping the deployment pipeline simple. I've been scratching my head a lot on Marten's migrations management workflow for the past few weeks but haven't been able to figure out to have something similar to what EF's modelbuilder provides (an other benefit being that it is easy to store and diff in source control). So deploying my application is quite a pain.
Right now, since I was big on using EF, then big on using CouchDb, since using marten this is the first time I need to figure out a deployment process to manually check and apply sql migrations, and I feel this is a huge step backward in my productivity. So in addition to performance and scalability, an additional goal pursued by the couchdb integration would be to keep the number of schema changes on the postgresql side very small.