I am trying to describe a 3D sliding gesture (only vertical or horizontal, without diagonals) on a given flat surface using so much conventional geometry or similar dissimilar learning techniques (the hidden Markov model is hidden, artificial neural networks, etc.) ) as possible. From the multiple observations of the data recovered from the device, I came to the conclusion that a hit can be described "easily" as a curve (or in some cases as a really straight line). With this question, I would like to know how you can describe a curve and a curved movement in simple geometric terms in the most efficient way (mainly speed, but also memory).
The publication is divided into two parts: one that provides information on the data used and another that provides an overview of what I have found so far. I am sorry in advance for my poor painting skills. :RE
3D position data
The device that I am using, transmits 3D points that represent the position of the hand at a given moment. I can capture and evaluate these. The following image displays the diagram of the data from two different perspectives: top to bottom and isometric (more or less):
- Flat view XY (on the left, aka top-down view) – for each sample only the values along the X and Y axes are taken into account. This view represents the surface of the device above which the movement of the hand is detected.
- XYZ view (on the right, aka isometric view) – for each sample, the three axes are taken into consideration. This view represents the complete 3D movement in a volume on the surface of the device that defines the space where gestures can be detected
In the following image I have added the movement of the hand as detected by the device:
The real movement looks more like this:
Based on the observation of the real movement and that detected by the device, I can mark almost half of the samples that the device has given me as invalid, that is, all the edge values (along each axis, one position can be between 0 and 65534), which does not describe the actual movement of the hand from the user's perspective of the device (in the image below, the invalid data is represented as the part of the trajectory that is covered by a polygon):
Of course, sometimes the "valid" part of the trajectory is quite small compared to the invalid data:
The algorithm I described below does not care how much of the valid data, as long as there are at least 2 samples that meet the requirement of not being edge positions, which means that X and Y are different from 0 and 65534. This is a problem. that I will detail in the next part of this post.
Describing the movement
I've thought about it a bit and this is what I came up with:
Extract only the set of valid samples that excludes all those that have an edge position
For each sample, generate a local XY coordinate system that is aligned with the XY coordinate system of the device surface (to make things easier :)):
Next, I'm thinking about calculating the vector between the current and the next sample (if present) and calculating the angle between that vector and the X axis (you can also do it with the Y axis):
Using the magnitude of each angle, I can determine if the movement between the current and the next sample is tilted more toward a horizontal or vertical one and also in which direction.
This should allow me to determine the general direction of the sliding movement, as well as the position on the surface. I have swept a lot: D but since I want to describe this in a more formal way, obviously I need to describe my findings, hence the need to find a way to describe and classify a curve according to its properties. Maybe calculate the curvature of the entire trajectory?
Of course, there are some problems with this algorithm that came to my mind:
I searched online before I started thinking about creating the algorithm I described earlier, but I could not find anything. Even the issue of curves classification seems not to be as popular or the search terms I used are too broad / restrictive. The classification here is not so essential (unlike what follows) but it would still be good to be able to divide the resulting curves into sets, each representing a sliding gesture.
The next thing I've been thinking about is the curve adjustment. I have read articles about this but, frankly, along with a couple of assignments at my university during the math course, I have not thought much about it, except Bezier's curves. Can anyone tell me if the adaptation of curves is a plausible solution for my case? Since it is curved suitable it would be wise to assume that we need an initial curve against which we want to make our adjustment. This would require gathering sliding movements and then extracting a possible optimal curve that is something like an "average" of all the curves for a given slip. I can use the first algorithm I described earlier to get a compact description of a curve and then store and analyze multiple curves for a given stroke to get the "perfect" curve. How do you proceed when handling the classification of curves?