The power of data analytics
This is why every organization engages artificial intelligence & machine learning. Once they have an “extensively trained” model that has a very good performance, they start selling it. Example: I tried Google Vision API to detect vehicle plates. The accuracy is amazing! Put many CCTVs on the roads, feed the image streams, predict the plate … Read moreGoogle Cloud Vision API to detect vehicle plate number
Imagine we want to recognize/identify an object in the images streamed from camera feeds (such as to recognize thief/suspect at the immigration checkpoint, airports, stations, etc.). To do that, the convolutional neural network (CNN) is currently the most used method. Such popular CNN architectures such as LeNet, AlexNet, VGG, GoogLeNet, ResNet, YOLO, etc. could be … Read moreCombining neural network and GPU in Google Cloud Platform
Two other popular data processing platform in the IT world are explored, i.e., DMX-h and SPSS Modeler. 1) DMX-h I was an extensive user of this beast software in 2009-2012. It is an amazing ETL platform, I used to process terabytes of chunked files which was completed in a short time (compared to a relational … Read moreSyncsort DMX-h & IBM SPSS Modeler
In the last post, I mentioned the Gartner’s magic quadrant as well as the competitive landscape of BI products. KDnuggets covers the data science & ML products in their article (https://bit.ly/2Pococi). Some interesting observations: 1) KNIME & Mathworks increases their completeness of vision in the last 3 years. KDNuggets quotes KNIME “With a wealth of … Read moreData science & ML (commercial) tools: their competitive landscape
Among many hyped agriculture technologies under industry 4.0 umbrella I’ve ever seen, this Microsoft solution, namely FarmBeats, appropriately addresses Indonesia’s geographical challenges. As a solution architect, what I like the most from the presentation is that how they synthesized the problem and designed a solution for it.Here are the summaries of the challenges:1) limited connectivity … Read morePrecision agriculture for a rural area with limited connectivity
On the last post, I mentioned that aggregating & sorting 100 million rows dataset (~ 2.4 GB) using monolithic approach takes 4 seconds to 5 minutes (R data.table, ptyhon pandas, awk, perl) to complete. Spark, a distributed platform that could be horizontally paralleled, takes almost 2 minutes. I extend the trial using Spark atop YARN … Read moreWhy we need a big data platform such as Hadoop & Spark?
Very often, many organizations insist to involve legacy resources (e.g., applications, data storage) into the big data system. On one hand, it could accelerate and ease the implementation of a big data use case, but it also creates a bottleneck in the workflow that would be problematic in the long term. If the monolithic applications … Read moreBe cautious to include legacy resources as part of the big data system