Unlike the stomach, the brain does not growl when it needs sustenance. It silently absorbs
what we provide, continuously shaping our thoughts, emotions, and actions. At Pradhi, we
recognize this silent hunger and believe in providing food for thought.
Welcome to Pradhi's Knowledge Hub. Here, we share what we learned from the books that
guided us and the people whose writings inspire us.
This is more than just a repository of information. It is a space designed to nurture
intellectual curiosity and foster continuous growth.
Our Knowledge Hub is built on the belief that learning is a lifelong journey. Each piece of
knowledge, whether it comes from a timeless book, a groundbreaking research paper, a
lecture series, or a conversation with an inspiring individual, adds a unique flavor
to our mental palate.
We invite you to join us on this journey.
Happy Learning
Food for thought
Blue Ocean Strategy
Boltzmann's Atom
Infinite Powers
Range: Why Generalists Triumph in a Specialized World
Quantum Mechanics
The Tell-Tale Brain
Phantoms in the Brain
Behave by Robert Sapolsky
Nuts!
The Language Instinct
Arthashastra
The Art of War
The book images displayed are for representation purposes only.
01
Word2Vec research paper
Mikolov et al. · Google · 2013
The Word2Vec research paper by Tomas Mikolov and his team at Google is a seminal work in the field of natural language processing. Published in 2013, this paper introduced a novel approach for creating word embeddings using neural networks, which has since become a cornerstone technique in machine learning and NLP. The Word2Vec model learns vector representations of words from large text corpora, capturing semantic relationships in a continuous vector space. By using two main architectures, Continuous Bag of Words (CBOW) and Skip-gram, the model efficiently predicts word contexts, significantly improving the performance of various NLP tasks. This groundbreaking research has paved the way for numerous advancements in understanding and processing human language with artificial intelligence.
02
Attention by Dmitri Bahdanau
Bahdanau, Cho & Bengio · Université de Montréal · 2014
Attention by Dmitri Bahdanau is a groundbreaking work in the field of artificial intelligence and natural language processing. Bahdanau introduced the attention mechanism in his influential paper "Neural Machine Translation by Jointly Learning to Align and Translate," co-authored with Kyunghyun Cho and Yoshua Bengio. This mechanism allows models to focus on relevant parts of the input sequence, significantly improving the performance of neural machine translation and other sequence-to-sequence tasks. The attention mechanism has since become a fundamental component in many advanced AI models, including transformers and BERT, revolutionizing the way machines understand and generate human language. This pioneering work has paved the way for numerous advancements in AI, making complex tasks more accurate and efficient.
03
Attention is All You Need
Vaswani et al. · Google · 2017
Attention is All You Need is a seminal research paper authored by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Published in 2017, this paper introduced the Transformer model, a revolutionary architecture in the field of natural language processing. Unlike traditional models that rely on recurrent or convolutional layers, the Transformer uses self-attention mechanisms to process sequences of data in parallel, allowing for more efficient training and improved performance on tasks such as translation, summarization, and text generation. The introduction of the Transformer has had a profound impact on AI research, leading to the development of powerful models like BERT, GPT, and T5, which continue to push the boundaries of what machines can understand and generate in human language.
04
LoRA: Low-Rank Adaptation of Large Language Models
Hu et al. · Microsoft · 2021
LoRA (Low-Rank Adaptation) is an innovative method developed to optimize the adaptation of large pre-trained language models for specific tasks. By introducing low-rank decomposition matrices into each layer of the Transformer architecture, LoRA significantly reduces the number of trainable parameters and computational requirements. This approach maintains high performance while minimizing the resource intensity typically associated with full model fine-tuning. LoRA's efficiency makes it a game-changer in the field of natural language processing, enabling broader accessibility and application of advanced language models across various domains.
05
A Mathematical Theory of Communication
Claude E. Shannon · Bell Labs · 1948
A Mathematical Theory of Communication by Claude E. Shannon, published in 1948, is a groundbreaking work that laid the foundation for modern information theory. Shannon introduced key concepts such as information entropy, which measures the uncertainty in a message, and channel capacity, the maximum rate of error-free data transmission. His work established the mathematical framework for encoding, transmitting, and decoding information efficiently, influencing the development of digital communication technologies, data compression, and error correction codes. Shannon's paper remains a cornerstone in the fields of telecommunications and computer science, shaping how information is processed and communicated globally.
06
Learning Representations by Back-Propagating Errors
Rumelhart, Hinton & Williams · 1986
Learning Representations by Back-Propagating Errors by David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams (1986) is a landmark paper in artificial intelligence. It introduced the backpropagation algorithm, a method that revolutionized the training of neural networks by efficiently calculating gradients needed for weight optimization through gradient descent. This breakthrough allowed multilayer neural networks to learn complex representations and solve non-linear problems, laying the groundwork for modern deep learning techniques used in various applications like image recognition and natural language processing. The paper remains foundational in the study and application of neural networks.
07
On the Opportunities and Risks of Foundation Models
Bommasani et al. · Stanford CRFM · 2021
On the Opportunities and Risks of Foundation Models by Rishi Bommasani and collaborators is a comprehensive examination of the paradigm shift in AI with the advent of large-scale models like BERT, GPT-3, and DALL-E. These models, trained on extensive datasets, are adaptable to a wide range of downstream tasks. The paper discusses the central yet incomplete nature of these "foundation models," highlighting their capabilities, technical principles, applications, and societal impacts. The report provides a detailed analysis of the models' capabilities across various domains such as language, vision, robotics, and human interaction. It also addresses the technical aspects, including model architectures, training procedures, and data management. Furthermore, the paper explores applications in critical fields like healthcare, law, and education, emphasizing both the potential benefits and the associated risks, such as ethical concerns, equity, and environmental impact.
08
The Kolmogorov-Arnold Networks (KAN)
Liu et al. · MIT · 2024
White Paper explores the theoretical foundation and applications of KANs, a powerful approach to modeling complex systems through neural networks. Inspired by the work of Andrey Kolmogorov and Vladimir Arnold, this architecture leverages their theorem on representing multivariate functions, enabling KANs to capture intricate, high-dimensional relationships with remarkable efficiency. The paper details the underlying mathematical principles, network structure, and potential use cases in fields like artificial intelligence, physics, and data science. By providing a robust framework for understanding non-linear dynamics, KANs offer a breakthrough in computational modeling and predictive analysis across diverse domains.
