Skip to main content
SpringerLink
Log in

A Few Shot Multi-representation Approach for N-Gram Spotting in Historical Manuscripts

  • Conference paper
  • First Online:
Frontiers in Handwriting Recognition (ICFHR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13639))

Included in the following conference series:

  • 1062 Accesses

Abstract

Despite recent advances in automatic text recognition, the performance remains moderate when it comes to historical manuscripts. This is mainly because of the scarcity of available labelled data to train the data-hungry Handwritten Text Recognition (HTR) models. The Keyword Spotting System (KWS) provides a valid alternative to HTR due to the reduction in error rate, but it is usually limited to a closed reference vocabulary. In this paper, we propose a few-shot learning paradigm for spotting sequences of a few characters (N-gram) that requires a small amount of labelled training data. We exhibit that recognition of important n-grams could reduce the system’s dependency on vocabulary. In this case, an out-of-vocabulary (OOV) word in an input handwritten line image could be a sequence of n-grams that belong to the lexicon. An extensive experimental evaluation of our proposed multi-representation approach was carried out on a subset of Bentham’s historical manuscript collections to obtain some really promising results in this direction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Almazan, J., Gordo, A., Fornés, A., Valveny, E.: Handwritten word spotting with corrected attributes. In: ICCV (2013)

    Google Scholar 

  2. Almazán, J., Gordo, A., Fornés, A., Valveny, E.: Word spotting and recognition with embedded attributes. IEEE TPAMI 36, 2552–2566 (2014)

    Article  Google Scholar 

  3. Antonacopoulos, A., Downton, A.C.: Special issue on the analysis of historical documents (2007)

    Google Scholar 

  4. Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. JMLR (2012)

    Google Scholar 

  5. Biswas, S., Banerjee, A., Lladós, J., Pal, U.: DocSegTr: an instance-level end-to-end document image segmentation transformer. arXiv preprint arXiv:2201.11438 (2022)

  6. Biswas, S., Riba, P., Lladós, J., Pal, U.: Beyond document object detection: instance-level segmentation of complex layouts. Int. J. Doc. Anal. Recogn. (IJDAR) 24(3), 269–281 (2021)

    Article  Google Scholar 

  7. Biswas, S., Riba, P., Lladós, J., Pal, U.: DocSynth: a layout guided approach for controllable document image synthesis. In: ICDAR (2021)

    Google Scholar 

  8. Bunke, H., Varga, T.: Off-line roman cursive handwriting recognition. In: Chaudhuri, B.B. (ed.) Digital Document Processing, pp. 165–183. Springer, London (2007). https://doi.org/10.1007/978-1-84628-726-8_8

    Chapter  Google Scholar 

  9. Choudhary, A., Rishi, R., Ahlawat, S.: A new character segmentation approach for off-line cursive handwritten words. Proc. Comput. Sci. 17, 88–95 (2013)

    Article  Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

    Google Scholar 

  11. Howe, N.R.: Part-structured inkball models for one-shot handwritten word spotting. In: ICDAR (2013)

    Google Scholar 

  12. Kang, L., Riba, P., Rusinol, M., Fornés, A., Villegas, M.: Distilling content from style for handwritten word recognition. In: ICFHR (2020)

    Google Scholar 

  13. Kang, L., Riba, P., Rusinol, M., Fornés, A., Villegas, M.: Content and style aware generation of text-line images for handwriting recognition. IEEE TPAMI (2021)

    Google Scholar 

  14. Kang, L., Riba, P., Wang, Y., Rusiñol, M., Fornés, A., Villegas, M.: GANwriting: content-conditioned generation of styled handwritten word images. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 273–289. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_17

    Chapter  Google Scholar 

  15. Konidaris, T., Kesidis, A.L., Gatos, B.: A segmentation-free word spotting method for historical printed documents. Pattern Anal. Appl. 19, 963–976 (2016)

    Article  MathSciNet  Google Scholar 

  16. Kozielski, M., Matysiak, M., Doetsch, P., Schlöter, R., Ney, H.: Open-lexicon language modeling combining word and character levels. In: ICFHR (2014)

    Google Scholar 

  17. Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350, 1332–1338 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  18. Lombardi, F., Marinai, S.: Deep learning for historical document analysis and recognition-a survey. J. Imaging 6, 110 (2020)

    Article  Google Scholar 

  19. Marcelli, A., Parziale, A., Senatore, R.: Some observations on handwriting from a motor learning perspective. In: AFHA (2013)

    Google Scholar 

  20. Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. IJDAR 5, 39–46 (2002)

    Article  MATH  Google Scholar 

  21. Parziale, A., Capriolo, G., Marcelli, A.: One step is not enough: a multi-step procedure for building the training set of a query by string keyword spotting system to assist the transcription of historical document. J. Imaging 6, 109 (2020)

    Article  Google Scholar 

  22. Poznanski, A., Wolf, L.: CNN-N-gram for handwriting word recognition. In: CVPR (2016)

    Google Scholar 

  23. Puigcerver, J., Toselli, A.H., Vidal, E.: Querying out-of-vocabulary words in lexicon-based keyword spotting. Neural Comput. Appl. 28, 2373–2382 (2017)

    Article  Google Scholar 

  24. Rath, T.M., Manmatha, R.: Word spotting for historical documents. IJDAR 9, 139–152 (2007)

    Article  Google Scholar 

  25. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NeurIPS (2015)

    Google Scholar 

  26. Sanchez, J.A., Toselli, A.H., Romero, V., Vidal, E.: ICDAR 2015 competition HTRtS: Handwritten text recognition on the transcriptorium dataset. In: ICDAR (2015)

    Google Scholar 

  27. Sauvola, J., Pietikäinen, M.: Adaptive document image binarization. Pattern Recogn. 33, 225–236 (2000)

    Article  Google Scholar 

  28. Shaffi, N., Hajamohideen, F.: Few-shot learning for Tamil handwritten character recognition using deep Siamese convolutional neural network. In: Mahmud, M., Kaiser, M.S., Kasabov, N., Iftekharuddin, K., Zhong, N. (eds.) AII 2021. CCIS, vol. 1435, pp. 204–215. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-82269-9_16

    Chapter  Google Scholar 

  29. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  30. Souibgui, M.A., Fornés, A., Kessentini, Y., Tudor, C.: A few-shot learning approach for historical ciphered manuscript recognition. In: ICPR (2021)

    Google Scholar 

  31. Stauffer, M., Fischer, A., Riesen, K.: Keyword spotting in historical handwritten documents based on graph matching. Pattern Recogn. 81, 240–253 (2018)

    Article  Google Scholar 

  32. Sudholt, S., Fink, G.A.: PHOCNet: a deep convolutional neural network for word spotting in handwritten documents. In: ICFHR (2016)

    Google Scholar 

  33. Vinciarelli, A., Luettin, J.: A new normalization technique for cursive handwritten words. Pattern Recogn. Lett. 22, 1043–1050 (2001)

    Article  MATH  Google Scholar 

  34. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: NeurIPS (2016)

    Google Scholar 

  35. Wang, T., Xie, Z., Li, Z., Jin, L., Chen, X.: Radical aggregation network for few-shot offline handwritten Chinese character recognition. Pattern Recogn. Lett. 125, 821–827 (2019)

    Article  Google Scholar 

  36. Wang, Y., Yao, Q., Kwok, J.T., Ni, L.M.: Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surv. (CSUR) 53, 1–34 (2020)

    Google Scholar 

  37. Wong, A., Yuille, A.L.: One shot learning via compositions of meaningful patches. In: ICCV (2015)

    Google Scholar 

Download references

Acknowledgment

This work has been partially supported by the Spanish projects RTI2018-095645-B-C21, PID2021-126808OB-I00 and FCT-19-15244, and the Catalan projects 2017-SGR-1783, the CERCA Program/Generalitat de Catalunya, PhD Scholarship from AGAUR (2021FIB-10010), and the DIEM Graduate Research Scholarship entitled “Strumenti di supporto alla trascrizione di documenti manoscritti di interesse storico-culturale”.

Author information

Authors and Affiliations

  1. DIEM - Department of Information and Electrical Engineering and Applied Mathematics, University of Salerno, Fisciano, Italy

    Giuseppe De Gregorio & Angelo Marcelli

  2. Computer Vision Center and Computer Science Department, Universitat Autònoma de Barcelona, Barcelona, Spain

    Sanket Biswas, Mohamed Ali Souibgui, Asma Bensalah, Josep Lladós & Alicia Fornés

Authors
  1. Giuseppe De Gregorio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanket Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Ali Souibgui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Asma Bensalah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Josep Lladós
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alicia Fornés
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Angelo Marcelli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanket Biswas .

Editor information

Editors and Affiliations

  1. Walmart Inc., Hoboken, NJ, USA

    Utkarsh Porwal

  2. Universitat Autònoma de Barcelona, Barcelona, Spain

    Alicia Fornés

  3. National University of Sciences and Technology (NUST), Islamabad, Pakistan

    Faisal Shafait

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

De Gregorio, G. et al. (2022). A Few Shot Multi-representation Approach for N-Gram Spotting in Historical Manuscripts. In: Porwal, U., Fornés, A., Shafait, F. (eds) Frontiers in Handwriting Recognition. ICFHR 2022. Lecture Notes in Computer Science, vol 13639. Springer, Cham. https://doi.org/10.1007/978-3-031-21648-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21648-0_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21647-3

  • Online ISBN: 978-3-031-21648-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation