PhD Study : Magic Hands: Hybrid Real / Synthetic Data Modelling of Natural Hand Motion and Gestures for Interactive, Immersive Virtual Reality.

Summary

Intensive rehabilitation is essential for stroke survivors but challenging to achieve due to limited access to rehabilitation therapy[1-3]. Our solutions use depth-sensing cameras to track hand movements and extract joint positions and orientations to model and identify hand gestures. Then, natural hand movement and position gestures are as game controllers for stroke survivors to interact effectively with rehab games. While initial successful research has been carried out to capture and analyse data captured from healthy hands, a significant research gap still exists accounting for disabled hands and variation in hand colour and size. Issues in detecting disabled hand gestures and movement include occlusion, position, the shape/pose of a disabled hand, and more limited movement articulation. In addition, a significant issue in creating the most effective models of hand position and motion is gathering enough data for Deep Learning algorithms to be effective.

This PhD will investigate existing and devise new state of the art Machine Learning algorithms to overcome the challenges outlined above, with a particular focus on Generative Adversarial Networks (GANs). The project will befit significantly from recent HERC funding to equip the new Automata Lab. Outcomes from the project will benefit from improved assistive technologies design, including improved useability of products from our recent spin-out company eXRt Intelligent Healthcare.

The proposed research will primarily focus on recognising disabled hand data. The method will include the collection of a novel, high-quality data set and supplementing this data with synthetic data using novel approaches. Various sensors and aesthetics for capturing hand and arm data will be investigated and compared, including webcams, depth cameras, haptics, and wi-fi sensors, focusing on those that are most precise. A redundancy architecture, including a novel, multi-resolution sensor fusion technique, will be considered for increased data tracking reliability. This system's robust development and refinement will require a balanced and representative data set of adequate volume. Whilst a new data set will be collected, it will be necessary to augment this real data to enhance system development accuracy and efficiency. Deep learning architectures such as GANs will be investigated and implemented for this purpose.

The research is fundamentally interdisciplinary, and as a team, we will work with physiotherapy colleagues in Ulster and at other global health technology research centres. We expect that the results of this PhD will support ongoing work in connected health and the real-time delivery of intelligent health services across Broadband and 5G.

This project will be an important case study within the upcoming CONFORM: CobOt eNvironments FOR Manufacturing project supported through HERC. This PhD project also builds upon prior Ulster research, including several European projects (EU H2020 projects MAGIC, VR-Relief, and MIDAS). Experimental work will utilise existing partnerships with the public and private sector, including the NI Stroke Network, the Stroke Association, and NI Brain Injury Matters. The project will avail state-of-the-art research infrastructure and equipment within the Intelligent Systems Research Centre (ISRC), including the Northern Ireland High-Performance Computing facility and the Spatial Computing and Neurotechnology Innovation Hub (SCAN i-hub).

Essential criteria

Applicants should hold, or expect to obtain, a First or Upper Second Class Honours Degree in a subject relevant to the proposed area of study.

We may also consider applications from those who hold equivalent qualifications, for example, a Lower Second Class Honours Degree plus a Master’s Degree with Distinction.

In exceptional circumstances, the University may consider a portfolio of evidence from applicants who have appropriate professional experience which is equivalent to the learning outcomes of an Honours degree in lieu of academic qualifications.

• Experience using research methods or other approaches relevant to the subject domain
• Research proposal of 1500 words detailing aims, objectives, milestones and methodology of the project
• A demonstrable interest in the research area associated with the studentship

Desirable Criteria

If the University receives a large number of applicants for the project, the following desirable criteria may be applied to shortlist applicants for interview.

• First Class Honours (1st) Degree
• Masters at 70%
• For VCRS Awards, Masters at 75%
• Experience using research methods or other approaches relevant to the subject domain
• Work experience relevant to the proposed project
• Publications - peer-reviewed
• Experience of presentation of research findings

Funding and eligibility

The University offers the following levels of support:

Vice Chancellors Research Studentship (VCRS)

The following scholarship options are available to applicants worldwide:

• Full Award: (full-time tuition fees + £19,000 (tbc))
• Part Award: (full-time tuition fees + £9,500)
• Fees Only Award: (full-time tuition fees)

These scholarships will cover full-time PhD tuition fees for three years (subject to satisfactory academic performance) and will provide a £900 per annum research training support grant (RTSG) to help support the PhD researcher.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Please note: you will automatically be entered into the competition for the Full Award, unless you state otherwise in your application.

Department for the Economy (DFE)

The scholarship will cover tuition fees at the Home rate and a maintenance allowance of £19,000 (tbc) per annum for three years (subject to satisfactory academic performance).

This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

• Candidates with pre-settled or settled status under the EU Settlement Scheme, who also satisfy a three year residency requirement in the UK prior to the start of the course for which a Studentship is held MAY receive a Studentship covering fees and maintenance.
• Republic of Ireland (ROI) nationals who satisfy three years’ residency in the UK prior to the start of the course MAY receive a Studentship covering fees and maintenance (ROI nationals don’t need to have pre-settled or settled status under the EU Settlement Scheme to qualify).
• Other non-ROI EU applicants are ‘International’ are not eligible for this source of funding.
• Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Due consideration should be given to financing your studies. Further information on cost of living