What should every programmer know about database design?

Mudassir Ali
Feb 10, 2020 06:01 AM 0 Answers
Member Since Dec 2019
Subscribed Subscribe Not subscribe
Mudassir Ali
- Feb 10, 2020 06:01 AM

Every programmer should be aware that there’s a world beyond RDBMS, SQL and the stuff the Joe and Jill Average use to build average apps.

How do we get to a data-driven environment? MySQL isn’t always the best starting point. But most people won’t bother to get their heads past relational databases for apps, which is too bad because some of the more powerful apps harness Hadoop, graph overlays for cross-web integration, KV or column stores for data streaming….

I guess that just means the ones who are willing to get their heads around more advanced techniques will reap the benefits. Here’s an example of what Bradford Cross did with FlightCaster a few years ago to do predictive analytics on data aggregated in a Hadoop cluster. He layered S3 -> EC2 -> Cloudera -> HDFS -> Hadoop -> Cascading -> Clojure.

Cross said, ” I’m not sure if those layers are exactly the right order, but you get the point. The key is go keep layering until you encapsulate the plumbing and get to the level of abstraction that lets you focus on solving your problem.” See
Nowhere in this interview is a “database” ever mentioned, but FlightCaster’s a mobile app that has predicted flight delays for years based on open data aggregated from various sources.

Cross a couple of years later with the help of some Stanford and UC Berkeley PhDs launched Prismatic, which mines social media with the help of ontologies and Clojure to feed its machine learning-based content recommendation engine:
Cross is just one example of a “programmer” who’s way beyond average.

Reply on This
Replying as Submit
0 Subscribers
Submit Answer
Please login to submit answer.
0 Answers