About LipiTk

Lipi Toolkit (LipiTk) is an open source toolkit for online Handwriting  Recognition (HWR), created by the Pen-based Applications and Handwriting Recognition Group at HP Labs India under the leadership of senior research scientist Sriganesh ("Sri-G") Madhvanath. The word lipi is Sanskrit for script.

The primary objective of LipiTk is to simplify the creation and deployment of HWR technology for new scripts. Other broad goals include supporting collaborative HWR R&D in academic and industrial settings, acting as a tool for user interface research, supporting commercial HWR development, promotion of standard ink representations and interfaces, promotion of sharing & reuse of tools, algorithms, code and handwriting datasets, and promotion of product and solution development.

Lipi Toolkit was first released internally in Oct 2005 and on SourceForge in Apr 2006.  Over the years we have added tools such as Lipi Designer to simplify creation of recognition engines for non-specialists, and Recognizers  for different scripts that application developers can download and use in their applications.  Various components of Lipi Toolkit are being used around the world today. 

More information regarding the history, design goals, related publications is available here.

Help spread the word !

If you find Lipi useful. You can use the images below (with a link to lipitk.sourceforge.net) in your applications:

Credits

HP Labs India

Sriganesh Madhvanath, Deepu Vijayasenan, K. Thanigai Murugan, Dinesh Mandalapu, Sridhar Muralikrishna, A. Bharath, Moteth Vijaya Kumar, Rajesh Pandry, Ba Hari Krishna, Srinivasa Vithal Charakana, Mudit Agrawal, V. Jagannadan, MNA Balaji, Ajay Patial, Jitender  Singh

Other Institutions

Jagadeesh Babu, L. Prashanth, Sri Sathya Sai University (SSSIHL), Puttaparthi - Implementing Digimemo-DCT, the data collection tool for ACECAD Digimemo
Indian Statistical Institute, Kolkata - Implementation of Neural Network classifier as part of the Core Toolkit
Niranjan Joshi, B.S.Raghavendra, C. K. Narayanan, Indian Institute of Science, Bangalore - First implementation of DTW classifier as part of the Core Toolkit
IRESTE, University of Nantes, France - Allowing the use of the IRONOFF handwriting database for training the Alphanum character recognizer
MILE lab, Indian Institute of Science, Bangalore - Hosting the HP Labs Handwriting Datasets

Other Contributions

Aravind Kumar - Lipik for Android - Code that automates copying of alphanumeric recognizer to the mobile device

Press & Awards

Lipi Toolkit wins Manthan Award 2007

The Manthan Award for India’s successful e-Content for Development 2007 was conferred on ‘ Lipi Toolkit ’ in the e-Localisation category. Lipi Toolkit is an open source toolkit for the creation of handwriting recognition (HWR) engines for local languages and scripts. The awards were given on September 22, 2007 at IICC, New Delhi and are given annually to promote usage of ICT enabled e-Content efforts at the grassroots level for innovation and ultimately e-Content for development in the country.

The Manthan Award is organized annually to scout and recognized some of the best practices in the field of e-Content based innovations. Launched in 2004, the award is organized by New Delhi based Digital Empowerment Foundation (DEF), in partnership with the World Summit on Information Society (WSIS). The winners of the awards are selected based on their contributions to bridge the digital divide and address the issues of grassroots-level empowerment and development. DEF has volunteered to play its part in organizing Manthan Award annually to bring out open best ICT and e-Content practices across India, by recognizing and felicitating the innovations and take them to a higher level of excellence. The winners of Manthan Award are selected to compete for World Summit Award under the auspices of the UN ICT system.

History

The need for a toolkit for online handwriting recognition was identified early in the Pen&HW work program at HP Labs India, towards the end of 2002. The main motivation was to simplify the creation of HWR engines for new scripts - both within the lab and collaborators outside -  so that the Indic and other neglected scripts in the developing world could be addressed much faster. Similar toolkits already existed for many other language technologies (e.g. Festival for Text-to-Speech Synthesis, Sphinx for Speech Recognition). As far as we were aware, no such toolkit existed for online HWR (perhaps with the exception of the now-defunct Npen from CMU which was focused on cursive English). One of the main challenges with the creation of such toolkits - especially in the case of HWR - is that no single set of techniques might work across all scripts. Unlike speech which is unidimensional and composed of a fix set of phonemes, scripts can vary considerably in structure, organization and complexity (see http://www.omniglot.com). In fact there is considerable variation within just the Indic scripts.

Nevertheless, there are certain steps that are central to the creation of data-driven HWR technology such as data collection, annotation and evaluation, and some algorithms such as Dynamic Time Warping (DTW) working on lowest-common-denominator features such as (x,y) coordinates have a good likelihood of success for simple shape matching across scripts. The shape recognition research at the lab on Tamil and Devanagari isolated characters led to the creation of some tools and generic shape-matching algorithms. In early 2004, we decided to start packaging these tools and algorithms into a first version of the toolkit that could be used for recognition of isolated shapes in other scripts.  An architecture for the toolkit was also created which supported custom configuration of the ICR engine and addition of new algorithms. Around this time, there was also interest from other teams within the lab looking for shape recognition capabilities for new shape sets (such as vowel diacritics) in the context of specific applications such as Gesture Keyboard.

Because of bandwidth constraints, an incremental approach has been adopted for the implementation of the toolkit. Commonly used objects such as Trace (ink) which are used by all of the tools, as well as the recognition algorithms have been identified, redefined and implemented using C++/STL. The existing recognition algorithms such as DTW have been rewritten to natively use the new definitions of these objects. Commonly used preprocessing functions (e.g. character size normalization) and utility functions (e.g. UNIPEN reader) have been collected in libraries, and training, testing and evaluation of recognizers facilitated by Perl scripts.

Early milestones

Research Publications

These are some publications that describe applications, standards, or techniques that are relevant to Lipi Toolkit.

GeCCo: Finger Gesture-based Command and Control for Touch Interfaces, Sriganesh Madhvanath, Dinesh Mandalapu, Tarun Madan, Naznin Rao, Ramesh Kozhissery. In 4th International Conference on Intelligent Human Computer Interaction (IHCI 2012), Kharagpur, India, Dec 2012

JollyMate: Assistive Technology for Young Children with Dyslexia, Jignesh Khakhar and Sriganesh Madhvanath. ICFHR 2010: 12th International Conference on Frontiers in Handwriting Recognition, Kolkata, India, Nov 2010

Online Handwriting Recognition for Indic Scripts, Bharath A. and Sriganesh Madhvanath. Book chapter in Indic OCR- Document Recognition and Retrieval (Springer). V. Govindaraju and S. Setlur (editors), 2009

A Framework for Adaptation of the Active-DTW Classifier for Online Handwritten Character Recognition, Roy Vandana, Madhvanath Sriganesh, Anand S, Sharma R R, 10th International Conference on Document Analysis and Recognition (ICDAR 2009), Barcelona, Spain, July 26 - 29, 2009

A Skew-tolerant Strategy and Confidence Measure for k-NN Classification of Online Handwritten Characters, Vandana Roy and Sriganesh Madhvanath, International Conference on Frontiers in Handwriting Recognition (ICFHR 2008).

Subspace-based and Dynamic Time Warping-based Methods for Online Handwritten Tamil Character Recognition, Deepu V. and Sriganesh Madhvanath. Vivek – Special Issue on Document Analysis, 16(2), pp 22-29, 2006

Elastic Matching of Online Handwritten Tamil and Telugu Scripts Using Local Features, Prashanth L., Jagadeesh Babu V., Raghunath Sharma R., Prabhakara Rao G.V., Dinesh Mandalapu, 9th International Conference on Document Analysis and Recognition (ICDAR 2007), Curitiba, Brazil, Sept 23-26, 2007

Active-DTW: A Generative Classifier that combines Elastic Matching with Active Shape Modeling for Online Handwritten Character Recognition, Sridhar, Muralikrishna; Mandalapu, Dinesh, Patel, Mehul, International Conference on Frontiers in Handwriting Recognition (ICFHR 2006) .

LipiTK: A Generic Toolkit for Online Handwriting Recognition: Sriganesh Madhvanath, Deepu Vijayasenan and Thanigai Murugan Kadiresan.International Workshop on Frontiers in Handwriting Recognition (IWFHR-10), La Baule, France, Oct 2006

Prototype Learning Methods for Online Handwriting Recognition, Raghavendra B.S., Narayanan C.K., Sita G., Ramakrishnan A.G., Sriganesh M., 8th International Conference on Document Analysis and Recognition (ICDAR 2005), Seoul, Korea, Aug 29 - Sept 1, 2005

Machine Recognition of Online Handwritten Devanagari Characters, Niranjan Joshi, G.Sita, A.G. Ramakrishnan, Deepu Vijayasenan, Sriganesh Madhvanath, 8th International Conference on Document Analysis and Recognition (ICDAR 2005), Seoul, Korea, Aug 29 - Sept 1, 2005

UPX: A New XML Representation for Annotated Datasets of Online Handwriting Data, Mudit Agrawal, Kalika Bali, Sriganesh Madhvanath and Louis Vuurpijl. In 8th International Conference on Document Analysis and Recognition (ICDAR 2005), Seoul, Korea, Aug-Sept 2005

Data Collection for Handwriting Corpus Creation in Indic Scripts, Mudit Agrawal, Ajay S Bhaskarabhatla and Sriganesh Madhvanath. In International Conference on Speech and Language Technology and Oriental COCOSDA (ICSLT-COCOSDA 2004), New Delhi, India, Nov 2004

Representation and Annotation of Online Handwritten Data, Ajay S. Bhaskarabhatla, Sriganesh Madhvanath, M. N. S. S. K. Pavan Kumar, A. Balasubramanian and C. V. Jawahar. In 9th International Workshop on Frontiers in Handwriting Recognition (IWFHR-9), Tokyo, Oct 2004

Online Handwriting Recognition for Tamil, H.K. Aparna, Vidhya Subramanian, Kasirajan, Vijay Prakash, V.S. Chakravarthy and Sriganesh Madhvanath. In 9th International Workshop on Frontiers in Handwriting Recognition (IWFHR-9), Tokyo, Oct 2004

Comparison of Elastic Matching Algorithms for Online Tamil Handwritten Character Recognition, Niranjan Joshi, G. Sita, and A. G. Ramakrishnan and Sriganesh Madhvanath. In 9th International Workshop on Frontiers in Handwriting Recognition (IWFHR-9), Tokyo, Oct 2004

Tamil Handwriting Recognition Using Subspace and DTW Based Classifiers, Niranjan Joshi, G. Sita, A. G. Ramakrishnan, Sriganesh Madhvanath. In 11th International Conference on Neural Information Processing (ICONIP 2004), Kolkata, India, Nov 2004

Elastic Matching Algorithms for Online Tamil Character Recognition, Niranjan Joshi, G. Sita, A. G. Ramakrishnan, Sriganesh Madhvanath. In 11th International Conference on Neural Information Processing (ICONIP 2004), Kolkata, India, Nov 2004

Principal Component Analysis for Online Handwritten Character Recognition, Deepu Vijayasenan and Sriganesh Madhvanath. In 17th International Conference on Pattern Recognition (ICPR 2004), Cambridge, United Kingdom, August 2004

Experiences in Collection of Handwriting Data for Online Handwriting Recognition in Indic Scripts, Ajay S Bhaskarabhatla and Sriganesh Madhvanath. In Fourth International Conference on Linguistic Resources and Evaluation (LREC 2004), Lisbon, Portugal, May 2004

An XML Representation for Annotated Handwriting Datasets for Online Handwriting Recognition, Ajay S Bhaskarabhatla and Sriganesh Madhvanath. In Fourth International Conference on Linguistic Resources and Evaluation (LREC 2004), Lisbon, Portugal, May 2004