The power of data analytics
Gartner’s magic quadrant is a well-known benchmark to check current competitive standing of a certain IT product. The report is released every year. Gartner maps the “ability to execute” on the y-axis and the “completeness of vision” on the x-axis. Then, they made 4 categories (a quadrant) based on their position, i.e., 1) Leaders (High … Read moreCompetition landscape of business intelligence products
Among many hyped agriculture technologies under industry 4.0 umbrella I’ve ever seen, this Microsoft solution, namely FarmBeats, appropriately addresses Indonesia’s geographical challenges. As a solution architect, what I like the most from the presentation is that how they synthesized the problem and designed a solution for it.Here are the summaries of the challenges:1) limited connectivity … Read morePrecision agriculture for a rural area with limited connectivity
There is no such a “Swiss Army knife” tool; every tool has its advantages in a certain circumstance, e.g., we know R has the most comprehensive statistical packages, but it also lacks scalability support. Python, another language, has tons of crowded discussion so that looking for a solution from the community is trivial. What if … Read morePolyglot data science applications
On the last post, I mentioned that aggregating & sorting 100 million rows dataset (~ 2.4 GB) using monolithic approach takes 4 seconds to 5 minutes (R data.table, ptyhon pandas, awk, perl) to complete. Spark, a distributed platform that could be horizontally paralleled, takes almost 2 minutes. I extend the trial using Spark atop YARN … Read moreWhy we need a big data platform such as Hadoop & Spark?
Very often, many organizations insist to involve legacy resources (e.g., applications, data storage) into the big data system. On one hand, it could accelerate and ease the implementation of a big data use case, but it also creates a bottleneck in the workflow that would be problematic in the long term. If the monolithic applications … Read moreBe cautious to include legacy resources as part of the big data system