Different Micro Realities

the concept of “the umwelt” coined by biologist Jakob von Uexküll in 1909 — the idea that different animals in the same ecosystem pick up on different elements of their environment and thus live in different micro-realities based on the subset of the world they’re able to detect. Eagleman stresses the importance of recognizing our own umwelt — our unawareness of the limits of our awareness:

I can see this in different ecosystems I observe – the entrepreneur ecosystem, the learning ecosystem and the microcosm of our own product teams and businesses.

In the startup ecosystem, different players – startups, investors, mentors and customers have different micro-realities about products, businesses, and markets. While they overlap somewhat, sometimes they are very different.

In the learning ecosystem similar micro-realities exist between institutions, teachers, students and learning experts.
But the best example, I have seen of these different micro-realities is in a product team or a business.

Source of inspiration

This Will Make You Smarter: 151 Big Thinkers Each Pick a Concept to Enhance Your Cognitive Toolkit

Notes from Data Science, Past, Present and Future

One of my favorite podcasts on Data Science is by Ben Lorica from O’Reilly. You can find more about the Data Show Podcast here.

This podcast is about Data Science Past, Present and Future – an interview with DJ Patil (a well-known Data Scientist). Here are a few snippets.

  1. Data Science is a big tent model. It requires a knowledge of Mathematics, Statistics, Programming,  a hacker mentality of exploration, and the ability to communicate well.
  2. Current Data Science applications are geared towards consumer internet. But there are lots of opportunities in areas like social sciences.
  3. A Data Driven Organization is one that uses data to test hypotheses and make decisions.
  4. Chief Data Officer (CDO) is responsible for the good stewardship of data in an organization. Sometimes, they are also known as CAO – Chief Analytics Officers
  5. Towards the end of the podcast, there was a discussion on data ethics, dilemma of decision making in automated systems and societal questions.

Early Signals

“What are you doing?”, my wife asked. Most of the time I grunt something. I think of this question as more of a greeting than a question. This time, however, I said enthusiastically “I am reading about the Wiki Wiki Web”.  This cracked up the entire family (two kids included). Wiki Wiki means quick in Hawaiian. This was in the early 90s before the days of Wikipedia. I was browsing through c2 wiki.

Wikis became mainstream when Wikipedia became popular and now my family does not laugh at me anymore.

There are times you have a hunch about certain trends. I felt strongly about – Databases in the 80s, Wikis and application components in 90s, XML, Python in early 2000 and now ML and Chatbots. This resulted in my working on an SQL database engine in the mid-80s, database components in 90s, an XML chip in the 2000s and on Python since 2006. Now, it is ML and chatbots and Natural Language Processing.

Not every thing I was excited about became mainstream. RDF and Semantic Web, OLE from Microsoft,  Domain Specific Languages and Pattern Oriented Languages did not go very far.

Over a period, I have built a few thumb rules about paying attention to early signals in emerging technologies.

  1. What is the research behind the technology and how long has it been going on? For example, neural networks and ML are several decades old. AI has gone through several difficulties.
  2. What is the volume and velocity of research papers?
  3. Is government funding research in this space? (Internet, Page Rank algorithm, self driving cars and many others started as funded Government research).
  4. Who are the major companies involved in the early adoption of these technologies? For XML it was Microsoft, Sun Microsystems, and several others.
  5. What are pilot projects being done for commercialization and who is working on those?
  6. Who is hiring in this space?
  7. Which business publications are covering topics about this space?
  8. What companies are getting funded? Funding is both a leading and lagging indicator depending on who is funding and why they are doing it.
  9. How is the information about this space being propagated? Who is propagating it?
  10. What are the conversations going on in Twitter?
  11. Are there books on the subject? Books are most of the time lagging indicators.
  12. Are these technology topics being covered in conferences?

Some of these indicators are easy to find. You need to look for others.

Restarting My Read Log

I am restarting my Read Log. A read log is a blog of a list of things I read and find useful. I tweet some of them but Tweets have a short half-life.

The inspiration for Read Log comes from different sources – Four short links by Nat, Brain Pickings from Maria, Farnam Street by Shane, and a few others.

Some of the best bloggers I know work hard at writing their posts and sometimes I feel like I am cheating. But these posts are worth sharing and if I am lucky, some of them may even start conversations.

Half Life of Knowledge

How long is your knowledge relevant? In other words, what is the half-life of your knowledge?

Wikipedia has a nice description of the half-life of knowledge

The half-life of knowledge or half-life of facts is the amount of time that has to elapse before half of the knowledge or facts in a particular area is superseded or shown to be untrue. These coined terms belong to the field of quantitative analysis of science known as scientometrics.

Here are a few things to think about:

  • What is the half-life of entrepreneur knowledge? Can we take lessons from the past and use them today?
  • What is the half-life of knowledge about software architecture and design?
  • What is the half-life of knowledge about sales and marketing techniques?

Some knowledge may have a shorter half-life, than others. To stay relevant in your industry you need to figure out how much of your knowledge is still useful.

Why I Retweet

There are several reasons. In no particular order:

  • I like the message in the tweet. I resonate with it. 
  • I like the link – typically a pointer to good reading material
  • Because it provides a different point of view 
  • I use it as a marker in my life – a part of my daily log
  • It is a hat tip to the author who causes me to pause and think. 
  • I think this tweet requires recognition and I would like to spread the idea
  • It may be a part of a discussion. I jump in and do my two bits.
  • It may be an event and I want to share it (a picture, a quote, a sound bite)
  • Same reason I tweet – to start a conversation
  • Same reason I tweet – to ask a question

Applied ML – How Uber Uses Machine Learning at Scale

This is one of the most comprehensive engineering blog posts on how Uber uses Machine Learning (ML) at scale. It covers:

  • Uber’s ML Platform – Michael Angelo
  • Uber’s research and production efforts and how they inter-relate
  • How Uber achieves Model Developers Velocity

I made a list of few terms and concepts from the article:

  • ML deployment use cases
  • Pervasive deployment of ML in several applications
  • Distributed training of ML
  • Aligning ML applications with Uber’s priorities
  • ML tools across the company (where and what)
  • Internal events like – ML conferences, ML reading groups, talk series
  • Data Science Workbench (a tool to build and iterate ML models)
  • ML Platform team and how they work to support ML development inside Uber
  • Technology stacks – Spark, Cassandra, Python and others
  • Experiments with external tools both open source and commercial
  • Uber’s open source contributions

It is nice to know how a dynamic company uses Machine Learning. There is a lot to learn from here. If you are thinking about building and deploying ML applications Scaling Machine Learning at Uber with Michelangelo | Uber Engineering Blog is a must read. I may go back and read it again.

A Great CTO Talk about Technology at Walmart

OrangeScape has this new initiative called CTO Talks. I think it is a brilliant idea. “While there are a lot of conversations taking place at the software development level, there are none at the CTO level”, says Suresh. I agree. We need different levels of conversations on technology.

I enjoyed  The talk on Technology at Walmart – a few Glimpses. I hope to see other more comprehensive blog posts. I was looking for the use of Machine Learning at Walmart, and I was not disappointed.

Here is a list of uses of Machine Learning (ML) at Walmart.

    • Competitive Intelligence and Analytics
    • Crawl frequency prediction (how frequently you can crawl certain sites for price information – too many crawls, and you will be blocked. Too few and you an miss useful information. Different sites update information at different intervals)
    • Natural Language Processing (NLP) of product catalogs
    • Bossa Nova robots roaming the aisles at Walmart locations checking out of stock items mislabeled shelf tags, and incorrect prices.
    • IOT  at Walmart – Monitoring temperatures of Refrigerators in real time
    • Visual inspection and Spoilage predictions
    • Predictive analytics of future failure of equipment
    • Predicting attrition (they have 2.3 million associates in over 11,000 locations)
    • Predicting absenteeism based on weather patterns an HR application (and an important contributor to maintaining service levels in their stores)
    • Hari briefly touched upon blockchain. They are looking at using it for tracing grocery items from source to customer.

Walmart is one of the leading indicators of technology adoption in retail. Hari mentioned that they were the first to introduce Satellites dishes in their store locations, barcode scanning, use of RFID and providing a direct view of store items to their suppliers.

It was a great talk. It was no wonder that we had an amazing turnout (more than 250 registrations). Hari answered all the questions patiently and in depth.

Discovering Information with a Little Help from Tweet Assistant

If you noticed a spike in my Twitter posts today, there is a reason.  I have been searching #machinelearning using Tweet Assistant (more on Twitter Assistant in a future post)

These tweets, hashtags, mentions and popular posts provide a gold mine of information.  However, there is more to analyzing tweets than getting news and opinions. Within minutes, I was following some cool people and started discovering awesome tweets and visualizations.

Machine Learning – Top 10 Mentions

 

Machine Learning – Top 5 Hashtags

I started retweeting some of the posts I found but felt that there are far too many posts and a discussion of these may fill a couple of blog posts. So that is what I am going to do.

 

 

Applications of ML – Improving Depth detection, Night Sight, Video Play and Notifications

Here are a few fascinating applications of ML. I mostly track business applications of ML so I was pleasantly surprised to see how Unsupervised Learning and Reinforcement Learning (two ML techniques that do not get much coverage) was being used by two of the biggies in the AI and ML space.

The first two – Depth detection and Nightsight are posts from Google AI Blog.  The following concepts were covered in these posts.

  • PSL Positive shutter lag
  • Motion metering
  • Exposure stacking
  • Astrophotography
  • Super-resolution
  • Auto white balancing and Learning based AWB algorithm
  • Ultra long shot photography
  • Optical Image stabilization

    Facebook Horizon article covered how they were improving user experience with Reinforcement Learning.

  • Improving video playing using Reinforcement Learning (RL)
  • Improving user notifications using Reinforcement Learning

Links to articles:

  1. Under the hood of the Pixel 2: How AI is supercharging hardware – Google AI 
  2. Google AI Blog: Night Sight: Seeing in the Dark on Pixel Phones
  3. Horizon: An open-source reinforcement learning platform – Facebook Code