Causal Learning, Transformers & Facebook's SEER

Top AI stories from last week

One of the purposes of AIM is to recognize and reward people & organizations in the Indian data science ecosystem. Each year, we come out with a sleuth of awards and lists of recognition that ultimately bring immense value and visibility to the whole industry. This is a serious pursuit we are after.

For eg, our recently published 40 under 40 data scientists awards were appreciated globally. We make sure our awards and lists are unbiased, unpaid and done under the supervision of an independent committee and with high rigour and research. These take a lot of efforts.

Nominations for  50 Best Firms In India For Data Scientists To Work For – 2021 is now open. This is an expanded list that is one of our most impactful each year. 

As part of Rising 2021 | our women in AI conference, we recognize the most influential women AI leaders in India. Nominations open here.

SKILLUP 2021 is one of its kind data science education fair scheduled on 22-23rd Apr.  AIM Data Science Faculty Excellence Awards recognizes and celebrates teachers and faculties in data science education on a global scale. Send your nominations here.

As part of SKILLUP, we are also organizing the blogging contest that urges data scientists to share their journey in the field. Nominations close on 25th Mar.

---

1

Yoshua Bengio & Why He Is Bullish About Causal Learning

Recently, Yoshua Bengio and researchers from the University of Montreal, the Max-Planck Institute for Intelligent Systems and Google Research demonstrated how causal representation learning contributes to the robustness and generalisation of machine learning models.

The causal model contains mechanisms that give rise to the observed statistical dependencies and allows the model of the distribution shifts through the notion of interventions. Also, causal relations can be viewed as the components of reasoning chains that provide predictions for situations very far from the observed distribution.

---

2

When Transformers Fail

Transformers are the de facto architecture of choice for natural language processing tasks. Since their introduction three years ago, Transformers have undergone several modifications.

Recently, a team of researchers from Google Research found that most modifications do not meaningfully improve transformers’ performance. Most of the Transformer variants found beneficial were either developed in the same codebase or are relatively minor changes.

---

3

IBM’s Strategy For Hybrid Cloud Growth In India

In a bid to accelerate its hybrid cloud strategy in India, IBM recently launched Cloud Satellite. Available across locations, the Cloud Satellite offers a standard set of cloud services complete with toolchains, databases, and AI.

In related news, IBM has received full Cloud Service Provider empanelment from the Indian Ministry of Electronics and IT (MeitY), allowing the tech giant to work with government agencies and other public sector undertakings. IBM is making inroads into India with its hybrid cloud strategy.

---

4

Featured Video | How AI is going to transform Creativity and more..

---

5

Now GPT-3 Gets A Performance Boost Of 30%

Recently, researchers from UC Berkeley, University of Maryland and UC Irvine showed the accuracy of the World’s largest language model, GPT-3 can be highly unstable across different prompts. They also developed a contextual calibration method, which improves the performance and accuracy of GPT-3 by up to 30%.

---

6

Hands-On Guides for ML Developers

+ PyTorch Code for Self-Attention Computer Vision

+ Guide to Google’s Tensor2Tensor for Neural Machine Translation

+ Introduction to Keras Graph Convolutional Neural Network(KGCNN) and Ragged Tensor

+ Guide to Scalable and Robust Bayesian Optimization with Dragonfly

+ What is Transformer XL?

---

7

PEOPLE & STARTUPS

+ Interview With Amit Deshpande: SpringML, Senior VP, India Development Center

+ Quality Requires Testing, Says Ramendeep Singh, Infogain

+ In Conversation With Vidushi Marda, Senior Programme Officer at ARTICLE-19

+ How This Delhi-based Startup Uses AI To Redefine Corporate Wellness

+ How SirionLabs Uses AI To Offer Contract Management Solutions

+ How This Techie’s Love For Storytelling Led Her To Start An AI-Driven Content Generation Startup

---

8

Facebook’s New Billion-Parameter Model

In self-supervised learning, systems don’t rely on labelled data sets to train and perform tasks. Instead, they learn directly from the information directly fed to them–text, images etc. Facebook vice president Yann LeCun presented the blueprint for self-supervised learning at the AAAI conference in 2020.

Now, with SEER (SElf-supERvised), Facebook has co-opted this approach for computer vision. SEER is a billion-parameter self-supervision computer vision model that can learn from any group of images on the internet. These images needn’t be curated and labelled, which are otherwise a prerequisite for most computer vision training.

---

9

The Most Advanced Neural Networks Discovered By OpenAI

In a major breakthrough, researchers at OpenAI have discovered neural networks within AI systems resembling the neural network inside the human brain. The multimodal neurons are one of the most advanced neural networks to date.

The researchers have found these advanced neurons can respond to a cluster of abstract concepts centred around a common high-level theme rather than a specific visual feature.

---

10

BOTTOM OF THE NEWS

Here's what all happened last week.

+ AMD Instinct MI100 Accelerators To Power AMPD Ventures’ ML Cloud Initiative

+ NVIDIA Unveils AI Enterprise Software Suite To Unleash The Power Of AI

+ Raspberry Pi To Get Some Fascinating Machine Learning Improvements

+ Redis Labs Announces General Availability for Integrated Enterprise Tiers of Azure Cache

+ Sanket Atal Appointed Salesforce’s New Managing Director For India