Bipost (beta) is a simple database synchronization tool built for developers in mind.
It is created to keep your databases on-premises while providing a way to extract and load specific sets of data to AWS Aurora.
It uses a similar ETL technique with recurring syncing in this order: Extract, Load & Transform.
How it works¶
- On every sync biPost.exe reads table schema's and data, creates a compressed file and uploads it to S3.
- You can customize query criteria to your source DB using customData.json file.
- Destination bucket triggers an API which reads schema's, creates database and tables (if they don't exist) and loads the data.
- Before and after data is loaded to MySQL you are able to call procedures and transform your data.
- On every upload MySQL uses a
REPLACEcommand so if you set the appropriate primary keys on source database, data would be updated in destination (and avoid duplicates).
- biPost.exe can run manually or automatically with a scheduled Windows Task.
Bipost API calls your RDS instance on your AWS account.
Each RDS Aurora instance loads data by accessing a dedicated bucket, exclusive to your AWS account.
- Bipost Sync is built with AWS SDK for .NET
- Upload to S3 is done through AWS SDK technology.
- S3 triggers an event and calls Bipost API built with Lambda.
- Lambda reads data and credentials and proceeds to connect to MySQL-Aurora.
- MySQL-Aurora is a managed database service, you don't have to worry about servers, software patching and other managing tasks.
- Aurora loads data from S3 dedicated bucket.
Done! Start building something amazing on top!
Check the Business Intelligence use case.
We are currently developing a secure web page to automate all settings that are exchanged over email. Thank you for your patience and we look forward to provide a world-class service.
0.4.2 (beta) 2017-09-16¶
- Table schemas are now synchronized against source definition on every sync, details here.
- Bipost.exe bugfixes.
- Bipost API bugfixes.
0.4.0 (beta) 2017-08-20¶
- Custom connections added.
- Initial statement added to API.
- Special characters are deleted on string columns of 100 characters length or up.