PhD Study : Development of Change Transformer—Attentive Deep-Learning Approaches for Abnormal Change Detection

Summary

Deep learning is a hot topic in machine learning and it is playing increasingly important roles in intelligence systems for our daily lives, such as computer vision, autonomous car driving, earth observation, etc. Deep learning is typically data driven, there is little domain knowledge utilised in training neural network models and determining decision boundaries. Learning process mainly focuses on searching and optimizing network models by gradient descent functions, without accounting for inherent discriminant and dependent characteristics of data distribution. Such a learning paradigm often makes it more difficult to achieve the convergence of training models, and therefore the resulting network models are not easy to generalize, typically in machine translation.

These observations have motivated the recent development of deep Transformer Networks. Transformer Networks make extensive use of attention mechanisms to discriminate the representative parts of data distribution based on contextual information and fade out the rest, thereby devoting more learning processes to deal with that small but representative part of the data for classification, recognition or prediction tasks. Currently ‘the two most common attention techniques used are dot-product attention, which uses the scalar product between vectors to highlight important parts of data, and multi-head attention, which uses combined attention to direct the overall attention of a network or sub-network’.

The proposed project will study existing attention techniques and develop new attention mechanisms that will be used as heuristics in a transformer network model. The project will incorporate one of two application scenarios into the development of attention:

1) detecting abnormal change from images, and

2) anomaly detection in electromagnetic satellite signals, which are part of the ongoing work undertaken in the supervisory team.

For the former case, the development of an attention mechanism could be inspired by human visual attention, in which humans often focus on an object of interest in an image with high resolution while perceiving the surrounding of the object at low resolution over time. Whereas for the latter case, as electromagnetic satellite signals might contain a range of events, not all parts of signals are equally relevant for a specific event. This observation could motivate development of an attention based on the contextual characteristics of an event to determine the relevant portions of signals. The transformer networks developed may be applicable to various real world application scenarios, such as monitoring abnormal singale change in Earth Observation, detecting faults in assemble lines of manufacturing processes, or human’s health deterioration in the context of healthcare.

Essential criteria

Applicants should hold, or expect to obtain, a First or Upper Second Class Honours Degree in a subject relevant to the proposed area of study.

We may also consider applications from those who hold equivalent qualifications, for example, a Lower Second Class Honours Degree plus a Master’s Degree with Distinction.

In exceptional circumstances, the University may consider a portfolio of evidence from applicants who have appropriate professional experience which is equivalent to the learning outcomes of an Honours degree in lieu of academic qualifications.

Desirable Criteria

If the University receives a large number of applicants for the project, the following desirable criteria may be applied to shortlist applicants for interview.

• First Class Honours (1st) Degree
• Masters at 70%
• Experience using research methods or other approaches relevant to the subject domain
• Work experience relevant to the proposed project
• Publications - peer-reviewed

Funding and eligibility

The University offers the following levels of support:

Vice Chancellors Research Studentship (VCRS)

The following scholarship options are available to applicants worldwide:

• Full Award: (full-time tuition fees + £19,000 (tbc))
• Part Award: (full-time tuition fees + £9,500)
• Fees Only Award: (full-time tuition fees)

These scholarships will cover full-time PhD tuition fees for three years (subject to satisfactory academic performance) and will provide a £900 per annum research training support grant (RTSG) to help support the PhD researcher.

Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Please note: you will automatically be entered into the competition for the Full Award, unless you state otherwise in your application.

Department for the Economy (DFE)

The scholarship will cover tuition fees at the Home rate and a maintenance allowance of £19,000 (tbc) per annum for three years (subject to satisfactory academic performance).

This scholarship also comes with £900 per annum for three years as a research training support grant (RTSG) allocation to help support the PhD researcher.

• Candidates with pre-settled or settled status under the EU Settlement Scheme, who also satisfy a three year residency requirement in the UK prior to the start of the course for which a Studentship is held MAY receive a Studentship covering fees and maintenance.
• Republic of Ireland (ROI) nationals who satisfy three years’ residency in the UK prior to the start of the course MAY receive a Studentship covering fees and maintenance (ROI nationals don’t need to have pre-settled or settled status under the EU Settlement Scheme to qualify).
• Other non-ROI EU applicants are ‘International’ are not eligible for this source of funding.
• Applicants who already hold a doctoral degree or who have been registered on a programme of research leading to the award of a doctoral degree on a full-time basis for more than one year (or part-time equivalent) are NOT eligible to apply for an award.

Due consideration should be given to financing your studies. Further information on cost of living