Skip to main content
SpringerLink
Log in

Automatic Sleep System Recommendation by Multi-modal RBG-Depth-Pressure Anthropometric Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a novel system for automatic sleep system recommendation using RGB, depth and pressure information. It consists of a validated clinical knowledge-based model that, along with a set of prescription variables extracted automatically, obtains a personalized bed design recommendation. The automatic process starts by performing multi-part human body RGB-D segmentation combining GrabCut, 3D Shape Context descriptor and Thin Plate Splines, to then extract a set of anthropometric landmark points by applying orthogonal plates to the segmented human body. The extracted variables are introduced to the computerized clinical model to calculate body circumferences, weight, morphotype and Body Mass Index categorization. Furthermore, pressure image analysis is performed to extract pressure values and at-risk points, which are also introduced to the model to eventually obtain the final prescription of mattress, topper, and pillow. We validate the complete system in a set of 200 subjects, showing accurate category classification and high correlation results with respect to manual measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Allen, B., Curless, B., & Popović, Z. (2004). Exploring the space of human body shapes: Data-driven synthesis under anthropometric control. Tech. rep., SAE Technical Paper

  • Azouz, Z. B., Shu, C., & Mantel, A. (2006). Automatic locating of anthropometric landmarks on 3d human models. In: Third International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 750–757. IEEE.

  • Bain, D., Ferguson-Pell, M., & McLeod, A. (2003). Evaluation of mattresses using interface pressure mapping. Journal of Wound Care, 12(6), 231–235.

    Article  Google Scholar 

  • Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4), 509–522.

    Article  Google Scholar 

  • Benjumea, A. C. (2001). Datos antropométricos de la población laboral española. Prevención, trabajo y salud: Revista del Instituto Nacional de Seguridad e Higiene en el Trabajo, 14, 22–30.

    Google Scholar 

  • Bookstein, F. L. (1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis & Machine Intelligence, 6, 567–585.

    Article  MATH  Google Scholar 

  • Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient n-d image segmentation. International Journal of Computer Vision, 70(2), 109–131.

    Article  Google Scholar 

  • Canda, A. S., & de Deportes, C. S. (2012). Variables antropométricas de la población deportista española. Consejo Superior de Deportes, Servicio de Documentación y Publicaciones.

  • Cippitelli, E., Gasparrini, S., Spinsante, S., & Gambi, E. (2015). Kinect as a tool for gait analysis: Validation of a real-time joint extraction algorithm working in side view. Sensors, 15(1), 1417–1434.

    Article  Google Scholar 

  • Clarkson, S., Wheat, J., Heller, B., & Choppin, S. (2014). Assessing the suitability of the microsoft kinect for calculating person specific body segment parameters. In Computer Vision-ECCV 2014 Workshops, pp. 372–385. Springer.

  • de Sanidad y Consumo, M. (2008). Estudio antropométrico de la población femenina española.

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Determann, R. M., Wolthuis, E. K., Spronk, P. E., Kuiper, M. A., Korevaar, J. C., Vroom, M. B., et al. (2007). Reliability of height and weight estimates in patients acutely admitted to intensive care units. Critical Care Nurse, 27(5), 48–55.

    Google Scholar 

  • DeVocht, J. W., Wilder, D. G., Bandstra, E. R., & Spratt, K. F. (2006). Biomechanical evaluation of four different mattresses. Applied Ergonomics, 37(3), 297–304.

    Article  Google Scholar 

  • Espitia-Contreras, A., Sanchez-Caiman, P., & Uribe-Quevedo, A. (2014). Development of a kinect-based anthropometric measurement application. In 2014 IEEE Virtual Reality (VR), pp. 71–72. IEEE.

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Foubert, N. (2010). Posture recognition and postural transition detection using bed-based pressure sensor. PhD thesis, Carleton University Ottawa.

  • Gordon, S. J., & Grimmer-Somers, K. (2011). Your pillow may not guarantee a good night’s sleep or symptom-free waking. Physiotherapy Canada, 63(2), 183–190.

    Article  Google Scholar 

  • Gordon, S. J., Grimmer-Somers, K., & Trott, P. (2009). Pillow use: The behaviour of cervical pain, sleep quality and pillow comfort in side sleepers. Manual Therapy, 14(6), 671–678.

    Article  Google Scholar 

  • Gordon, S. J., Grimmer-Somers, K. A., & Trott, P. H. (2010). Pillow use: The behavior of cervical stiffness, headache and scapular/arm pain. Journal of Pain Research, 3, 137.

    Google Scholar 

  • Gupta, S., Markey, M. K., & Bovik, A. C. (2010). Anthropometric 3d face recognition. International Journal of Computer Vision, 90(3), 331–349.

    Article  Google Scholar 

  • Harada, T., Mori, T., Nishida, Y., Yoshimi, T., & Sato, T. (1999). Body parts positions and posture estimation system based on pressure distribution image. In Proceedings of the 1999 IEEE international conference on robotics and automation (Vol. 2, pp. 968–975). IEEE.

  • Harada, T., Sato, T., & Mori, T. (2001). Pressure distribution image based human motion tracking system using skeleton and surface integration model. In Proceedings 2001 ICRA IEEE international conference on robotics and automation (Vol. 4, pp. 3201–3207). IEEE.

  • Huang, S. H., & Pan, Y. C. (2014). Ergonomic job rotation strategy based on an automated rgb-d anthropometric measuring system. Journal of Manufacturing Systems, 33(4), 699–710.

    Article  Google Scholar 

  • Huang, W., Wai, A.A.P., Foo, S.F., Biswas, J., Hsia, C.C., & Liou, K. (2010). Multimodal sleeping posture classification. In 2010 20th international conference on pattern recognition (ICPR), pp. 4336–4339. IEEE.

  • Joint, F., Organization, W. H., et al. (1985). Energy and protein requirements: report of a joint fa.

  • Jonker, R., & Volgenant, A. (1987). A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing, 38(4), 325–340.

    Article  MathSciNet  MATH  Google Scholar 

  • Kim, H. S., Park, K. H., & Jeoung, J. W. (2013). Can we measure the intraocular pressure when the eyeball is against the pillow in the lateral decubitus position? Acta Ophthalmologica, 91(7), e502–e505.

    Article  Google Scholar 

  • Lazzaro, E., Mallick, A., Singh, M., Reich, I., Elmann, S., Stefanov, D. G., et al. (2014). The effect of positional changes on intraocular pressure during sleep in patients with and without glaucoma. Journal of Glaucoma, 23(5), 282–287.

    Article  Google Scholar 

  • Leilnahari, K., Fatouraee, N., Khodalotfi, M., Sadeghein, M. A., & Kashani, Y. A. (2011). Spine alignment in men during lateral sleep position: experimental study and modeling. Biomedical Engineering Online, 10(1), 103.

    Article  Google Scholar 

  • Liu, J.J., Xu, W., Huang, M.C., Alshurafa, N., Sarrafzadeh, M., Raut, N., & Yadegar, B. (2013). A dense pressure sensitive bedsheet design for unobtrusive sleep posture monitoring. In 2013 IEEE international conference on pervasive computing and communications (PerCom), pp 207–215. IEEE.

  • Liu, J. J., Xu, W., Huang, M. C., Alshurafa, N., Sarrafzadeh, M., Raut, N., et al. (2014). Sleep posture analysis using a dense pressure sensitive bedsheet. Pervasive and Mobile Computing, 10, 34–50.

    Article  Google Scholar 

  • López-Torres, M., Porcar, R., Solaz, J., & Romero, T. (2008). Objective firmness, average pressure and subjective perception in mattresses for the elderly. Applied Ergonomics, 39(1), 123–130.

    Article  Google Scholar 

  • Lorenz, M. W., Graf, M., Henke, C., Hermans, M., Ziemann, U., Sitzer, M., et al. (2007). Anthropometric approximation of body weight in unresponsive stroke patients. Journal of Neurology, Neurosurgery & Psychiatry, 78(12), 1331–1336.

    Article  Google Scholar 

  • Lorenzo-Navarro, J., Castrillón-Santana, M., & Hernández-Sosa, D. (2013). On the use of simple geometric descriptors provided by rgb-d sensors for re-identification. Sensors, 13(7), 8222–8238.

    Article  Google Scholar 

  • Madadi, M., Escalera, S., Gonzalez, J., Roca, F. X., & Lumbreras, F. (2015). Multi-part body segmentation based on depth maps for soft biometry analysis. Pattern Recognition Letters, 56, 14–21.

    Article  Google Scholar 

  • Martinez, M., & Stiefelhagen, R. (2014). Kinect unbiased. In 2014 IEEE International Conference on Image Processing (ICIP), pp. 5791–5795. IEEE.

  • Matsuo, J., Sugama, J., Sanada, H., Okuwa, M., Nakatani, T., Konya, C., et al. (2011). Development and validity of a new model for assessing pressure redistribution properties of support surfaces. Journal of Tissue Viability, 20(2), 55–66.

    Article  Google Scholar 

  • Metsis, V., Kosmopoulos, D., Athitsos, V., & Makedon, F. (2014). Non-invasive analysis of sleep patterns via multimodal sensor input. Personal and Ubiquitous Computing, 18(1), 19–26.

    Article  Google Scholar 

  • Miller, S., Parker, M., Blasiole, N., Beinlich, N., & Fulton, J. (2013). A prospective, in vivo evaluation of two pressure-redistribution surfaces in healthy volunteers using pressure mapping as a quality control instrument. Ostomy/Wound Management, 59(2), 44–48.

    Google Scholar 

  • Mogelmose, A., Bahnsen, C., Moeslund, T.B., Clapés, A., & Escalera, S. (2013). Tri-modal person re-identification with rgb, depth and thermal features. In 2013 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 301–307. IEEE.

  • Nechala, P., Mahoney, J., & Farkas, L. G. (1999). Digital two-dimensional photogrammetry: A comparison of three techniques of obtaining digital photographs. Plastic and Reconstructive Surgery, 103(7), 1819–1825.

    Article  Google Scholar 

  • Nguyen, T. V., Feng, J., & Yan, S. (2014). Seeing human weight from a single rgb-d image. Journal of Computer Science and Technology, 29(5), 777–784.

    Article  Google Scholar 

  • Reyes, M., Clapés, A., Ramírez, J., Revilla, J. R., & Escalera, S. (2013). Automatic digital biometry analysis based on depth maps. Computers in Industry, 64(9), 1316–1325.

    Article  Google Scholar 

  • Romero Collazos, J. (2008). Análisis de la forma y la proporcionalidad. antropometria aplicada a la nutrición.

  • Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG), 23(3), 309–314.

    Article  Google Scholar 

  • Rusu, R. B. (2013). Semantic 3D object maps for everyday robot manipulation (Vol. 85). Heidelberg: Springer.

  • Sáenz, Z. L., Arias, A., Guzmán, E. C., & Arias, d L. (2011). Analysis of ergonomics conditions of a brand of mattress and pillows. University-industry project, medellin-colombia. Work (Reading, Mass), 41, 1281–1287.

    Google Scholar 

  • Suzuki, K., Horiba, I., & Sugie, N. (2003). Linear-time connected-component labeling based on sequential local operations. Computer Vision and Image Understanding, 89(1), 1–23.

  • Uhm, T., Park, H., & Park, J. I. (2015). Fully vision-based automatic human body measurement system for apparel application. Measurement, 61, 169–179.

    Article  Google Scholar 

  • Verhaert, V., Druyts, H., Van Deun, D., Berckmans, D., Verbraecken, J., Vandekerckhove, M., Haex, B., & Vander Sloten, J. (2011a). The use of a generic human model to personalize bed design. In: Proceedings of 1st international symposium on digital human modeling, vol 2202. Lyon, June, paper ID.

  • Verhaert, V., Druyts, H., Van Deun, D., De Wilde, T., Van Brussel, K., Haex, B., et al. (2011b). Modeling human-bed interaction: The predictive value of anthropometric models in choosing the correct bed support. Work (Reading, Mass), 41, 2268–2273.

    Google Scholar 

  • Verhaert, V., Haex, B., Wilde, T. D., Berckmans, D., Verbraecken, J., Valck, Ed, et al. (2011c). Ergonomics in bed design: the effect of spinal alignment on sleep parameters. Ergonomics, 54(2), 169–178.

  • Wang, Q., Jagadeesh, V., Ressler, B., & Piramuthu, R. (2014). Im2fit: Fast 3d model fitting and anthropometrics using single consumer depth camera and synthetic data. arXiv:1410.0745.

  • Wong, M., Lai, A., Singh, M., & Chew, P. (2013). Sleeping posture and intraocular pressure. Singapore Medical Journal, 54(3), 146–148.

    Article  Google Scholar 

  • Yang, C., Mao, Y., Cheung, G., Stankovic, V., & Chan, K. L. (2014). Graph-based depth video denoising and event detection for sleep monitoring. In 2014 IEEE 16th international workshop on multimedia signal processing (MMSP), IEEE, pp 1–6.

  • Yousefi, R., Ostadabbas, S., Faezipour, M., Nourani, M., Tamil, L., & Pompeo, M. (2011). Posture and limb detection for pressure ulcer prevention. In Proceedings of the Southern Biomedical Engineering Conference (SBEC).

  • Yu, M. C., Wu, H., Liou, J. L., Lee, M. S., & Hung, Y. P. (2012). Breath and position monitoring during sleeping with a depth camera. In HEALTHINF, pp 12–22.

  • Zuberi, N. A., Rekab, K., & Nguyen, H. V. (2004). Sleep apnea avoidance pillow effects on obstructive sleep apnea syndrome and snoring. Sleep and Breathing, 8(4), 201–207.

    Article  Google Scholar 

Download references

Acknowledgments

This work has been partially supported by Spanish Project TIN2013-43478-P and Dormity.com®.

Author information

Authors and Affiliations

  1. Computer Vision Center, Campus UAB, Edifici O, 08193, Cerdanyola del Vallès, Barcelona, Spain

    Cristina Palmero, Pouya Ahmadmonfared & Sergio Escalera

  2. Servei Universitari de Recerca en Fisioteràpia (S.U.R.F), Escola Universitària Gimbernat, Avinguda de la Generalitat, 202-206, 08174, Sant Cugat del Vallès, Barcelona, Spain

    Jordi Esquirol, Vanessa Bayo & Miquel Àngel Cos

  3. Dormity.com®, Via Augusta, 85-87, 08174, Sant Cugat del Vallès, Barcelona, Spain

    Joan Salabert & David Sánchez

  4. Dept. Matemàtica Aplicada i Anàlisi, UB, Gran Via de les Corts Catalanes 585, 08007, Barcelona, Spain

    Sergio Escalera

Authors
  1. Cristina Palmero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jordi Esquirol
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vanessa Bayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Miquel Àngel Cos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pouya Ahmadmonfared
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joan Salabert
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sergio Escalera
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cristina Palmero.

Additional information

Communicated by Hiroshi Ishikawa, Takeshi Masuda, Yasuyo Kita and Katsushi Ikeuchi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Palmero, C., Esquirol, J., Bayo, V. et al. Automatic Sleep System Recommendation by Multi-modal RBG-Depth-Pressure Anthropometric Analysis. Int J Comput Vis 122, 212–227 (2017). https://doi.org/10.1007/s11263-016-0919-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0919-0

Keywords

Search

Navigation