Summary

Intensive rehabilitation is essential for stroke survivors but challenging to achieve due to limited access to rehabilitation therapy[1-3]. Our solutions use depth-sensing cameras to track hand movements and extract joint positions and orientations to model and identify hand gestures. Then, natural hand movement and position gestures are as game controllers for stroke survivors to interact effectively with rehab games. While initial successful research has been carried out to capture and analyse data captured from healthy hands, a significant research gap still exists accounting for disabled hands and variation in hand colour and size. Issues in detecting disabled hand gestures and movement include occlusion, position, the shape/pose of a disabled hand, and more limited movement articulation. In addition, a significant issue in creating the most effective models of hand position and motion is gathering enough data for Deep Learning algorithms to be effective.

This PhD will investigate existing and devise new state of the art Machine Learning algorithms to overcome the challenges outlined above, with a particular focus on Generative Adversarial Networks (GANs). The project will befit significantly from recent HERC funding to equip the new Automata Lab. Outcomes from the project will benefit from improved assistive technologies design, including improved useability of products from our recent spin-out company eXRt Intelligent Healthcare.

The proposed research will primarily focus on recognising disabled hand data. The method will include the collection of a novel, high-quality data set and supplementing this data with synthetic data using novel approaches. Various sensors and aesthetics for capturing hand and arm data will be investigated and compared, including webcams, depth cameras, haptics, and wi-fi sensors, focusing on those that are most precise. A redundancy architecture, including a novel, multi-resolution sensor fusion technique, will be considered for increased data tracking reliability. This system's robust development and refinement will require a balanced and representative data set of adequate volume. Whilst a new data set will be collected, it will be necessary to augment this real data to enhance system development accuracy and efficiency. Deep learning architectures such as GANs will be investigated and implemented for this purpose.

The research is fundamentally interdisciplinary, and as a team, we will work with physiotherapy colleagues in Ulster and at other global health technology research centres. We expect that the results of this PhD will support ongoing work in connected health and the real-time delivery of intelligent health services across Broadband and 5G.

This project will be an important case study within the upcoming CONFORM: CobOt eNvironments FOR Manufacturing project supported through HERC. This PhD project also builds upon prior Ulster research, including several European projects (EU H2020 projects MAGIC, VR-Relief, and MIDAS). Experimental work will utilise existing partnerships with the public and private sector, including the NI Stroke Network, the Stroke Association, and NI Brain Injury Matters. The project will avail state-of-the-art research infrastructure and equipment within the Intelligent Systems Research Centre (ISRC), including the Northern Ireland High-Performance Computing facility and the Spatial Computing and Neurotechnology Innovation Hub (SCAN i-hub).


Essential criteria

Applicants should hold, or expect to obtain, a First or Upper Second Class Honours Degree in a subject relevant to the proposed area of study.

We may also consider applications from those who hold equivalent qualifications, for example, a Lower Second Class Honours Degree plus a Master’s Degree with Distinction.

In exceptional circumstances, the University may consider a portfolio of evidence from applicants who have appropriate professional experience which is equivalent to the learning outcomes of an Honours degree in lieu of academic qualifications.

  • Experience using research methods or other approaches relevant to the subject domain
  • Research proposal of 1500 words detailing aims, objectives, milestones and methodology of the project
  • A demonstrable interest in the research area associated with the studentship

Desirable Criteria

If the University receives a large number of applicants for the project, the following desirable criteria may be applied to shortlist applicants for interview.

  • First Class Honours (1st) Degree
  • Masters at 70%
  • For VCRS Awards, Masters at 75%
  • Experience using research methods or other approaches relevant to the subject domain
  • Work experience relevant to the proposed project
  • Publications - peer-reviewed
  • Experience of presentation of research findings

Funding and eligibility

The University offers the following levels of support:

Vice Chancellors Research Studentship (VCRS)

Full award (full-time PhD fees + DfE level of maintenance grant + RTSG for 3 years).

This scholarship will cover full-time PhD tuition fees and provide the recipient with £15,840 (tbc) maintenance grant per annum for three years (subject to satisfactory academic performance). This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Vice-Chancellor’s Research Bursary (VCRB)

Part award (full-time PhD fees + 50% DfE level of maintenance grant + RTSG for 3 years).

This scholarship will cover full-time PhD tuition fees and provide the recipient with £8,000 maintenance grant per annum for three years (subject to satisfactory academic performance). This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Vice-Chancellor’s Research Fees Bursary (VCRFB)

Fees only award (PhD fees + RTSG for 3 years).

This scholarship will cover full-time PhD tuition fees for three years (subject to satisfactory academic performance). This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Department for the Economy (DFE)

The scholarship will cover tuition fees at the Home rate and a maintenance allowance of £15,840 (tbc) per annum for three years (subject to satisfactory academic performance). This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

  • Candidates with pre-settled or settled status under the EU Settlement Scheme, who also satisfy a three year residency requirement in the UK prior to the start of the course for which a Studentship is held MAY receive a Studentship covering fees and maintenance.
  • Republic of Ireland (ROI) nationals who satisfy three years’ residency in the UK prior to the start of the course MAY receive a Studentship covering fees and maintenance (ROI nationals don’t need to have pre-settled or settled status under the EU Settlement Scheme to qualify).
  • Other non-ROI EU applicants are ‘International’ are not eligible for this source of funding.
  • Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Due consideration should be given to financing your studies. Further information on cost of living


Recommended reading

[1] A. Stephenson et al., ‘Evaluation of the Acceptability and Usability of the MAGIC-GLASS Virtual Reality Solution as Part of the Care Pathway in People with Acute, Sub-Acute and Chronic Stroke: A Study Protocol’, Physical Therapy Reviews, vol. 25, no. 2, pp. 118–127, Apr. 2020, doi: 10.1080/10833196.2020.1757379.

[2] M. McNeill, D. Charles, J. Burke, J. Crosbie, and S. McDonough, ‘Evaluating User Experiences in Rehabilitation Games’, Journal of Assistive Technologies, vol. 6, no. 3, pp. 173–181, Jul. 2012, doi: 10.1108/17549451211261290.

[3] S. Howes, I. M. Wilson, K. Pedlow, D. Holmes, D. K. Charles, and S. M. McDonough, ‘Older adults’ experience of active computer gaming for falls prevention exercise: A mixed-methods study, Physiotherapy Practice and Research, pp. 1–11, Aug. 2021, doi: 10.3233/ppr-210544.

[4] S. Howes, D. K. Charles, K. Pedlow, I. Wilson, D. Holmes, and S. McDonough, ‘User-centred design of an active computer gaming system for strength and balance exercises for older adults, Journal of Enabling Technologies, vol. 13, no. 2, pp. 101–111, Jun. 2019, DOI: 10.1108/JET-12-2018-0057.

[5] S. Howes, D. K. Charles, D. Holmes, K. Pedlow, I. Wilson, and S. McDonough, ‘Older adults’ experience of falls prevention exercise delivered using active gaming and virtual reality, Physiotherapy, vol. 103, no. Supplement 1, pp. E4–E5, Dec. 2017, DOI: 10.1016/j.physio.2017.11.155.

[6] S. C. Howes, D. K. Charles, J. Marley, K. Pedlow, and S. M. McDonough, ‘Gaming for Health: Systematic Review and Meta-analysis of the Physical and Cognitive Effects of Active Computer Gaming in Older Adults’, Physical Therapy, vol. 97, no. 12, pp. 1122–1137, Dec. 2017, doi: 10.1093/ptj/pzx088.

[7] D. E. Holmes, D. K. Charles, P. J. Morrow, S. McClean, and S. M. McDonough, ‘Leap motion controller and oculus rift virtual reality headset for upper arm stroke rehabilitation’, in Virtual Reality, Nova Science Publishers, Inc., 2017, pp. 83–102. Accessed: Nov. 25, 2021. [Online]. Available: http://www.scopus.com/inward/record.url?scp=85034771955&partnerID=8YFLogxK

[8] D. Holmes, D. Charles, P. Morrow, S. McClean, and S. M. McDonough, ‘Using Fitt’s Law to Model Arm Motion Tracked in 3D by a Leap Motion Controller for Virtual Reality Upper Arm Stroke Rehabilitation.: The 29th International Symposium on Computer-Based Medical Systems (CBMS 2016)’, Aug. 2016, DOI: 10.1109/CBMS.2016.41.

[9] D. Holmes, D. K. Charles, P. Morrow, S. McClean, and S. M. McDonough, ‘Usability and performance of leap motion and oculus rift for upper arm virtual reality stroke rehabilitation’, Journal of Alternative Medicine Research, vol. 9, no. 4, pp. 1–10, Feb. 2017.

[10] B. Cowley and D. Charles, ‘Behavlets: a method for practical player modelling using psychology-based player traits and domain-specific features’, User Modelling and User-Adapted Interaction, vol. 26, pp. 257–306, Jun. 2016, DOI: 10.1007/s11257-016-9170-1.

[11] D. Charles, K. Pedlow, S. M. McDonough, K. Shek, and T. Charles, ‘Close range depth-sensing cameras for virtual reality-based hand rehabilitation’, Journal of Assistive Technologies, vol. 8, no. 3, pp. 138–149, Sep. 2014, DOI: 10.1108/JAT-02-2014-0007.


The Doctoral College at Ulster University