2015 was an intensive and productive year in Base. Before heading off to new challenges, we wanted to sum up and give you a glimpse into what we’ve shipped.
Just to be clear, this post is about technology but the one thing we never ever forget at Base Lab is that the technology we build serves a purpose. And the impact of the technology we build is really something.
If you sum up all the activity time of Base users this year the total will come up to over 570 years. And this is counting our web interface only! Over 570 years of human productivity in 2015 alone. Wow. Now, coming back to technology – there are many things to write about but we selected a few that we think stand out and I’m trying to explain what they meant.
Good front door – our API
With more and larger sales teams adopting Base for their productivity, the need to be able to truly connect our product with the external world rose to the top of our priorities at the beginning of 2015.
We decided to really invest in it and do it the right way by implementing the API Facade Design Pattern, which is a programmatic proxy for microservices and API management written in Go. It’s the front door for all requests. It provides security, dynamic routing, load distribution, traffic management, usage monitoring and analytics. Every request is recorded and analyzed for anomalies using our Big Data stack.
All the API calls must be authenticated. Gatekeeper, our identity management service acts as an OAuth 2.0 provider and issues access tokens for client applications so they can act on behalf of Base users. The same service implements SAML 2.0 protocol exposing Single Sign-On functionality.
The whole API project wouldn’t be useful without documentation. We document every service in Markdown, automatically transform it into JSON Hyper Schema which is a machine-readable format and which in turn is used to generate the content of developers.getbase.com website, official client libraries in four languages (Ruby, Python, PHP and Java) and serves as API discovery.
PaaS for easy integrations
Having a great API ties neatly with the fact that we want to provide an environment for our customers and external developers to build custom functionalities that integrate easily with Base services. To this end we’ve built the first version of a new PaaS where external developers can contribute API-based applications without having direct access to our internal tools and servers. The PaaS is based on well known and developer friendly solutions like Docker or GitHub.
1 second resolution in infrastructure monitoring
Additionally, visibility was definitely one of our highest priorities on the infrastructure side. We now collect system metrics, such as various network and I/O statistics, memory and CPU usage with 1 second resolution. This gives us unprecedented visibility into anomalies and short spikes in resource usage that otherwise disappear when measured at longer intervals.
This best in class, fully replicated setup collects over 350,000 data points per second and is a foundation of our alerting and monitoring dashboards. We have over 15TB of active metrics on SSD drives, 500GB of in-memory cache and 10TB of archived data.
On top of that a separate flow based on Fluentd and Kibana supports our log collection and analysis, digesting information from applications, databases and system-level logs.
Data oriented from the ground up
The idea that data is the foundation for all businesses and that next generation of business software needs to be data oriented from the ground up is not news to us.
We started 2015 with significant elements of our own big data platform already in place. The data infrastructure we’ve built allows us to collect large amounts of high granularity data into a single, scalable data pipeline enabling us to effectively process it in both batch and real-time modes. Integrating all data – from users behaviour, through all backend operations and finally capturing every change in data objects gives us the basis to build valuable data products and fuels our data-driven mindset.
Email intelligence that wowed our customers
On top of these capabilities we’ve built our first major functionalities that benefit our users. First came Time Capsule, which is a historical exporter that allows our users to export snapshots of their sales pipelines from any point in time, real time reporting and intelligent close date suggestions. Then Email Intelligence really wowed our customers. It analyzes and scores all incoming emails in real-time and notifies our users about transactions that might be in danger, allowing them to take action as soon as possible and fix the situation. In this case we’re especially proud of how our stack allowed us to quickly move from prototyping a model to a scalable and reliable data product deployed to production.
700% performance boost and more
Part of being successful is that it tends to surface bigger and bigger challenges. Our iOS app experienced that too when we noticed significant performance issues on accounts with very large amounts of data. We took a deep look into the data model and were able to achieve over 700% performance boost with regard to creating and saving objects to the database, together with up to 50% decrease in database size on disk.
To display large datasets while keeping the UI responsive we elaborated our own mechanism based on NSFetchedResultsController, but working on a background thread and communicating with the main thread asynchronously – as opposed to Apple’s default pattern of working with fetched results controller on main thread.
All this not only speeds up the first synchronization from the web backend, but also greatly improves general experience of the application during daily use. All in all in 2015 we released 19 new versions of the iOS app, not counting patch releases – this is a new release in less than every 3 weeks! – and made 22 updates to the Core Data data model while preserving the ability to gracefully migrate from any model version to the newest one (our data model is well over 100 entities right now).
The other side of the great mobile divide saw big steps forward too. With over 40k installations in 2015 and an average star rating of 4.3 our Android app fell into “Excellent” category and received Top Developer designation on Google Play. All this happened while producing 33 feature releases and 19 bugfix releases. This means an average of a new version release for every week. To top this app off we’ve made several releases of our open source projects – the most popular one being FloatingActionButton by Jerzy which amassed over 3000 stars on GitHub.
So these are some of the things we’ve built in 2015. We don’t want to dwell in the past though. We’re already off looking into the future and working on things that will make an impact for our customers in 2016. And thanks to an additional $30M investment we attracted from top Silicon Valley investors in September our focus is both on accelerating the growth of our customer base and on continuing to build functionalities that create the new paradigm of business software and redefine the way millions of people will work in the future.