handposeutils package
Module Contents
handposeutils.calculations package
- handposeutils.calculations.geometry.get_cross_finger_angles(pose) dict[str, float][source]
Measure the angle between direction vectors of adjacent fingers.
- Parameters:
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Mapping from adjacent finger name pairs (e.g., “THUMB-INDEX”) to angle in radians.
- Return type:
dict of str to float
- handposeutils.calculations.geometry.get_finger_curvature(finger_name: str, pose) float[source]
Estimate the average angular curvature of a finger.
Curvature is measured as the mean angle (in radians) between consecutive segment vectors. Lower values indicate a straighter finger.
- Parameters:
finger_name (str) – Name of the finger (case-insensitive).
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Average curvature angle in radians.
- Return type:
float
- handposeutils.calculations.geometry.get_finger_length(finger_name: str, pose) float[source]
Calculate the total 3D length of a finger.
Highkey, this information is pretty arbitrary, since unless you normalize scaling, absolute distances aren’t particularly useful, compared to relative distances. TODO: write a relative bone length function. That would be more useful.
The length is computed by summing the Euclidean distances between each pair of adjacent joints for the given finger.
- Parameters:
finger_name (str) – Name of the finger (case-insensitive). Must be a key in handposeutils.data.constants.FINGER_MAPPING (e.g., “thumb”, “index”, “middle”, “ring”, “pinky”).
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Total finger length in the same units as the pose coordinates.
- Return type:
float
- handposeutils.calculations.geometry.get_finger_segment_lengths(finger_name: str, pose) list[float][source]
Get the individual segment lengths of a finger.
Returns the lengths of the proximal, intermediate, and distal segments by computing Euclidean distances between successive joints.
- Parameters:
finger_name (str) – Name of the finger (case-insensitive). Must be a key in handposeutils.data.constants.FINGER_MAPPING.
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Three lengths (same units as coordinates), ordered from base to tip.
- Return type:
list of float
See also
get_finger_lengthgets the summed length of the full finger
- handposeutils.calculations.geometry.get_finger_spread(pose) dict[str, float][source]
Measure the angular spread between adjacent fingers at their MCP joints.
- Parameters:
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Mapping from finger pair names (e.g., “INDEX-MIDDLE”) to spread angle in radians.
- Return type:
dict of str to float
- handposeutils.calculations.geometry.get_hand_aspect_ratio(pose) float[source]
Calculate the aspect ratio (width / height) of the hand in the XY plane.
- Parameters:
pose (Pose) – Hand pose object
- Returns:
Ratio of hand width to height in the XY plane.
- Return type:
float
- handposeutils.calculations.geometry.get_joint_angle(triplet: tuple[int, int, int], pose) float[source]
Compute the internal angle at the middle joint of a 3-point chain.
- Parameters:
triplet (tuple of int) – Landmark indices (a, b, c) where b is the vertex joint.
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Angle at point b in radians.
- Return type:
float
- handposeutils.calculations.geometry.get_palm_normal_vector(pose) ndarray[source]
Compute the palm normal vector using three base landmarks.
Uses the cross product of the wrist-to-index_mcp and wrist-to-pinky_mcp vectors to obtain the palm’s outward normal.
- Parameters:
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Normalized 3D vector representing the palm normal.
- Return type:
numpy.ndarray
- handposeutils.calculations.geometry.get_pose_flatness(pose, axis='z') float[source]
Measure the flatness of the hand pose along a given axis.
Flatness is defined as the standard deviation of all coordinates along the specified axis.
- Parameters:
pose (Pose) – Hand pose object with get_all_coordinates() method.
axis ({'x', 'y', 'z'}, optional) – Axis along which to compute flatness. Default is ‘z’.
- Returns:
Standard deviation along the specified axis. Lower values mean flatter pose.
- Return type:
float
- handposeutils.calculations.geometry.get_total_hand_span(pose) float[source]
Compute the Euclidean distance between thumb tip and pinky tip.
- Parameters:
pose (Pose) – Hand pose object or sequence supporting index-based access to Coordinate objects.
- Returns:
Distance between landmarks 4 (thumb tip) and 20 (pinky tip).
- Return type:
float
- handposeutils.calculations.geometry.vector_between(c1, c2)[source]
Returns a NumPy vector from Coordinate c1 to c2.
- Parameters:
c1 (Coordinate) – First point (x,y,z)
c2 (Coordinate) – Second point (x,y,z)
- Returns:
A NumPy array representing the vector from c1 to c2.
- Return type:
np.array
- handposeutils.calculations.similarity.compute_joint_angle_errors(pose1: HandPose, pose2: HandPose) List[float][source]
Computes per-joint absolute angle differences between two hand poses.
This method assumes the standard 21-landmark MediaPipe format. Angles are measured in radians for each consecutive finger joint triplet.
- handposeutils.calculations.similarity.cosine_similarity(pose1: HandPose, pose2: HandPose) float[source]
Compute cosine similarity between two 3D hand poses.
Each pose is represented as a flattened 63-dimensional vector (21 landmarks × 3 coordinates). The cosine similarity measures the angular difference between the vectors, making it invariant to scale but not to translation, so position is first normalized.
- Parameters:
- Returns:
similarity – Cosine similarity in the range [-1, 1]. - 1.0 indicates identical orientation. - 0.0 indicates orthogonal poses. - -1.0 indicates opposite orientation.
- Return type:
float
Notes
This method normalizes translation by centering poses before comparison.
Similarity is undefined for zero-length pose vectors (returns 0.0).
- handposeutils.calculations.similarity.embedding_similarity(vec1: ndarray, vec2: ndarray, method: str = 'cosine', **kwargs) float[source]
Computes similarity or distance between two embedding vectors or sequences.
Supports: - Single embeddings (1D arrays) - Temporal embeddings (2D arrays of shape [sequence_length, embedding_dim]),
where similarity is computed per frame and averaged.
- Parameters:
vec1 (numpy.ndarray) – First embedding vector or sequence.
vec2 (numpy.ndarray) – Second embedding vector or sequence.
method (str, default="cosine") – Method to compute similarity. Options: - ‘cosine’ - ‘euclidean’ - ‘manhattan’ - ‘mahalanobis’
**kwargs –
Additional parameters for specific methods. For example: - cov : numpy.ndarray
Covariance matrix for Mahalanobis distance.
- Returns:
The method name and the computed similarity or distance score. For cosine similarity, higher is more similar. For distances, lower is more similar.
- Return type:
tuple of (str, float)
- Raises:
ValueError – If vectors have different shapes, or covariance matrix shape is invalid.
NotImplementedError – If the given method is not supported.
- handposeutils.calculations.similarity.euclidean_distance(pose1: HandPose, pose2: HandPose) float[source]
Compute the mean Euclidean distance between two hand poses.
This function calculates the average distance between corresponding landmarks of two HandPose objects. Distances are computed directly in 3D space and are sensitive to both scale and translation.
It is STRONGLY recommended to normalize HandPoses before computing Euclidean distance.
- Parameters:
- Returns:
mean_distance – Mean Euclidean distance between corresponding landmarks. Lower values indicate greater similarity.
- Return type:
float
- Raises:
ValueError – If the number of landmarks (or their dimensionality) differs between poses.
- handposeutils.calculations.similarity.joint_angle_similarity(pose1: HandPose, pose2: HandPose) float[source]
Computes biomechanical similarity between two hand poses using joint angles.
The joint angles of each pose are extracted and compared using mean squared difference. Lower values indicate more similar poses.
- Parameters:
- Returns:
Mean squared difference between joint angles. A value of 0.0 indicates identical angles.
- Return type:
float
- Raises:
ValueError – If the angle descriptors have different lengths.
Notes
Useful in determining similarity in joint curvature across regions of the hand,
when used consecutively. - Can represent the same information as geometry.get_finger_curvature() when multiple function calls are fused.
See also
geometry.get_finger_curvature
- handposeutils.calculations.similarity.pose_similarity(pose1: HandPose, pose2: HandPose, method: str = 'procrustes') float[source]
Computes similarity between two hand poses using the specified method.
Supported methods
‘procrustes’: Procrustes distance (lower = more similar)
‘euclidean’ : Euclidean distance
‘cosine’ : Cosine similarity
‘joint_angle’: Mean squared joint angle difference
- param pose1:
First hand pose.
- type pose1:
HandPose
- param pose2:
Second hand pose.
- type pose2:
HandPose
- param method:
Similarity computation method.
- type method:
str, default=’procrustes’
- returns:
Similarity score according to the chosen method. Scale and interpretation vary depending on the method.
- rtype:
float
- raises NotImplementedError:
If the given method is not supported.
- handposeutils.calculations.similarity.procrustes_alignment(pose1: HandPose, pose2: HandPose) Tuple[ndarray, ndarray, float][source]
Perform Procrustes alignment between two 3D hand poses.
The alignment process removes translation, scale, and rotation differences between two HandPose objects, returning their aligned coordinates and the Procrustes distance (sum of squared differences).
It is STRONGLY recommended to normalize HandPoses before computing Euclidean distance.
- Parameters:
- Returns:
aligned_pose1 (ndarray of shape (n_landmarks, 3)) – Aligned version of pose1 after Procrustes transformation.
aligned_pose2 (ndarray of shape (n_landmarks, 3)) – Normalized version of pose2 for comparison.
distance (float) – Procrustes distance between aligned poses. Lower values indicate greater similarity.
- Raises:
ValueError – If the number of landmarks (or their dimensionality) differs between poses.
Notes
This method uses the Kabsch algorithm to compute the optimal rotation.
The Procrustes distance is not invariant to landmark correspondence errors.
- handposeutils.calculations.transforms.mirror_pose(pose: HandPose, axis: Literal['x', 'y', 'z'] = 'x') HandPose[source]
Mirrors a hand pose across the specified axis.
- handposeutils.calculations.transforms.normalize_handpose(pose: HandPose) HandPose[source]
Normalizes a hand pose’s position and scale.
This function applies: 1. Translation so the centroid is at the origin. 2. Uniform scaling to fit in [-1, 1] on all axes.
- handposeutils.calculations.transforms.normalize_handpose_positioning(pose: HandPose) HandPose[source]
Translates a hand pose so that its centroid is at the origin.
Each coordinate is shifted such that the average x, y, and z positions across all landmarks are zero.
- handposeutils.calculations.transforms.normalize_handpose_scaling(pose: HandPose) HandPose[source]
Scales a hand pose to fit within a [-1, 1] cube across all axes.
The scaling is uniform and based on the largest axis range (max range of x, y, or z coordinates).
- Parameters:
pose (HandPose) – The hand pose to scale.
- Returns:
The uniformly scaled hand pose, preserving proportions.
- Return type:
Notes
If the maximum range is zero (all points are identical), the pose is returned unchanged.
- handposeutils.calculations.transforms.rotate_pose_by_axis(pose: HandPose, degrees: float, axis: Literal['x', 'y', 'z']) HandPose[source]
Rotates a hand pose by a given angle around a specified axis.
- handposeutils.calculations.transforms.straighten_finger(pose, finger: str) HandPose[source]
Straightens a specified finger so that its joints align in a straight line.
The finger’s base and first joint define the direction, and each joint is repositioned proportionally along this direction, preserving original segment lengths.
- Parameters:
pose (HandPose) – The hand pose containing the finger to straighten.
finger (str) – Finger name to straighten. Must match a key in handposeutils.data.constants.FINGER_MAPPING.
- Returns:
The modified hand pose with the finger straightened.
- Return type:
- Raises:
ValueError – If the finger name is invalid or has fewer than two joints.
handposeutils.data package
- class handposeutils.data.coordinate.Coordinate(x: float, y: float, z: float)[source]
Bases:
object- normalize() Coordinate[source]
- scale(scalar: float) Coordinate[source]
- x: float
- y: float
- z: float
- class handposeutils.data.data_reader.DataReader[source]
Bases:
object- static convert_HandPoseSequence_to_json(sequence: HandPoseSequence, fps: int = 30) Dict[source]
Convert a HandPoseSequence into a JSON-serializable dictionary containing timed hand poses.
Output format: {
“fps”: int, # frames per second “sequence”: [
- {
“start_time”: float, “end_time”: float, “pose”: { … } # HandPose JSON format
]
}
- Parameters:
sequence (HandPoseSequence) – Sequence of timed hand poses.
fps (int, optional) – Frames per second metadata (default is 30).
- Returns:
JSON-compatible dictionary representing the sequence.
- Return type:
dict
- static convert_HandPose_to_mediapipe(pose: HandPose)[source]
Convert a HandPose object to MediaPipe NormalizedLandmarkList format.
Coordinates are converted directly without scaling (expected normalized).
- Parameters:
pose (HandPose) – Hand pose to convert.
- Returns:
MediaPipe landmarks list matching the pose’s coordinates.
- Return type:
mediapipe.framework.formats.landmark_pb2.NormalizedLandmarkList
- static convert_HandPose_to_openpose(pose: HandPose) List[float][source]
Convert a HandPose object to OpenPose flat keypoint format.
Output format is a list: [x0, y0, c0, x1, y1, c1, …, x20, y20, c20] with confidence c_i set to 1.0 by default.
- Parameters:
pose (HandPose) – Hand pose to convert.
- Returns:
Flat list representing OpenPose keypoints.
- Return type:
List[float]
- static convert_csv_to_HandPose(df: DataFrame, side='right_hand') HandPose[source]
Convert a CSV DataFrame of hand landmark coordinates to a HandPose object.
Expected CSV format: DataFrame with columns ‘x’, ‘y’, and ‘z’, with one row per landmark in order.
- Parameters:
df (pandas.DataFrame) – DataFrame containing columns [‘x’, ‘y’, ‘z’] for 21 hand landmarks.
side (str, optional) – Hand side label, default is ‘right_hand’.
- Returns:
Converted hand pose with coordinates from the DataFrame.
- Return type:
- static convert_json_to_HandPose(json_data: Dict[str, Any]) HandPose[source]
Convert JSON-formatted hand landmarks to a HandPose object.
- Parameters:
json_data (dict) –
JSON dictionary representing a hand pose, expected format: {
”side”: “right_hand”, # optional, default to ‘right_hand’ “landmarks”: [
{“x”: float, “y”: float, “z”: float}, # 21 landmarks …
]
} Some formats may nest landmarks under a “pose” key: {
- ”pose”: {
“landmarks”: […]
}, “side”: “left_hand”
}
- Returns:
HandPose instance with landmarks converted from JSON.
- Return type:
- static convert_json_to_HandPoseSequence(json_data: Dict[str, Any]) HandPoseSequence[source]
Convert a JSON dictionary representing a timed hand pose sequence into a HandPoseSequence object.
Expected JSON format: {
- “sequence”: [
- {
“start_time”: float, “end_time”: float, “pose”: { … } # HandPose JSON format, see export_HandPose_to_json
]
}
Example
- {
- “sequence”: [
- {
“start_time”: 0.0, “end_time”: 0.033, “pose”: { … } // HandPose JSON – reference convert_HandPose_to_json
}, {
“start_time”: 0.033, “end_time”: 0.066, “pose”: { … }
}
]
}
- Parameters:
json_data (dict) – JSON dictionary representing a sequence of timed hand poses.
- Returns:
Sequence object containing timed hand poses.
- Return type:
See also
convert_json_to_HandPoseruns on each pose found in the HandPoseSequence
HandPose,HandPoseSequence
- static convert_mediapipe_to_HandPose(mp_landmarks, handedness: str = None) HandPose[source]
Convert MediaPipe landmarks to a HandPose object.
MediaPipe landmarks are normalized 3D coordinates with x, y in [0,1] and z roughly in [-0.5, 0.5]. This function scales x and y by 100, inverts y axis, and scales z by 100.
- Parameters:
mp_landmarks (mediapipe.framework.formats.landmark_pb2.NormalizedLandmarkList) – MediaPipe landmarks object containing 21 hand landmarks.
handedness (str, optional) – ‘left’ or ‘right’ to indicate hand side (default None).
- Returns:
The converted hand pose with scaled coordinates.
- Return type:
Notes
The coordinate system is transformed so y is flipped vertically.
- static convert_openpose_to_HandPose(openpose_data: List[float], side=None) HandPose[source]
Convert OpenPose flat hand keypoint data to a HandPose object.
OpenPose format is a flat list of floats: [x0, y0, c0, x1, y1, c1, …, x20, y20, c20] where xi, yi are 2D coordinates and ci is confidence (ignored). Depth (z) is set to 0.0 by default.
- Parameters:
openpose_data (List[float]) – Flat list with 63 elements (21 points × 3 values).
side (str, optional) – Hand side label (‘left_hand’ or ‘right_hand’), default None.
- Returns:
Hand pose with 21 landmarks converted from OpenPose data.
- Return type:
- static export_HandPose_to_csv(pose: HandPose) DataFrame[source]
Export a HandPose object to a CSV-format pandas DataFrame.
Output DataFrame columns: ‘name’ (landmark name), ‘x’, ‘y’, ‘z’ (coordinates).
- Parameters:
pose (HandPose) – Hand pose to export.
- Returns:
DataFrame with one row per landmark and columns [‘name’, ‘x’, ‘y’, ‘z’].
- Return type:
pandas.DataFrame
- static export_HandPose_to_json(pose: HandPose) Dict[source]
Export a HandPose object to a JSON-serializable dictionary.
Output format: {
“side”: “right_hand”, “name”: “pose_name”, # optional pose identifier “landmarks”: [
- {“x”: float, “y”: float, “z”: float,
“name”: str, “finger”: str}, # 21 landmarks with metadata
…
]
}
Example
- {
“side”: “right_hand”, “landmarks”: [
{ “x”: 0.1, “y”: 0.2, “z”: 0.0, “name”: “WRIST”, “finger”: “PALM” }, { “x”: 0.15, “y”: 0.22, “z”: 0.0, “name”: “THUMB_CMC”, “finger”: “THUMB” }, …
]
}
- Parameters:
pose (HandPose) – HandPose to convert.
- Returns:
JSON-compatible dictionary describing the hand pose.
- Return type:
dict
- static save_frames_to_folder(sequence: HandPoseSequence, folder_name: str, file_prefix: str, handpose_prefix_name: str, verbose: bool = True)[source]
Save each frame of a HandPoseSequence as an individual JSON file.
Files are saved as: {folder_name}/{file_prefix}_{frame_index}.json
Each file contains: {
“start_time”: float, “end_time”: float, “pose”: { … } # HandPose JSON
}
- Parameters:
sequence (HandPoseSequence) – The sequence of timed hand poses to save.
folder_name (str) – Path to the directory to save JSON files (created if missing).
file_prefix (str) – Prefix for filenames, e.g., ‘frame’ produces files like ‘frame_1.json’.
handpose_prefix_name (str) – Prefix to assign to HandPose.name for each saved frame.
verbose (bool, optional) – Whether to print progress messages (default True).
- Return type:
None
- class handposeutils.data.handpose.HandPose(coordinates: List[Coordinate], side: Literal['left_hand', 'right_hand'], name: str = None)[source]
Bases:
objectRepresents a single 3D hand pose with 21 landmarks in MediaPipe format.
Each landmark is stored along with its coordinate, side (left or right hand), common anatomical name, and the finger grouping it belongs to.
- Parameters:
coordinates (list of Coordinate) – A list of exactly 21 Coordinate objects representing the hand’s landmarks.
side ({'left_hand', 'right_hand'}) – The handedness of the pose.
name (str, optional) – An optional identifier for the pose.
- Raises:
ValueError – If coordinates does not contain exactly 21 elements.
- get_all_coordinates() List[Coordinate][source]
Get all coordinates for this hand pose.
- Returns:
All 21 coordinates in index order.
- Return type:
list of Coordinate
- get_coordinate_by_index(index: int) Coordinate[source]
Retrieve a coordinate by its landmark index.
- Parameters:
index (int) – The landmark index (0–20).
- Returns:
The coordinate object at the specified index.
- Return type:
- get_handedness() Literal['left_hand', 'right_hand'][source]
Get the handedness of the pose.
- Returns:
The handedness of this pose.
- Return type:
{‘left_hand’, ‘right_hand’}
- get_index_by_common_name(name: str) int[source]
Get the landmark index for a given common anatomical name.
- Parameters:
name (str) – The common name of the landmark (e.g., “WRIST”, “INDEX_FINGER_TIP”).
- Returns:
The index (0–20) of the landmark.
- Return type:
int
- Raises:
ValueError – If no landmark with the given name is found.
- mirror(axis: Literal['x', 'y', 'z'] = 'x') HandPose[source]
Mirror the pose across a specified axis.
- Parameters:
axis ({'x', 'y', 'z'}, default='x') – The axis to mirror across.
- Returns:
A new mirrored HandPose instance.
- Return type:
- normalize() HandPose[source]
Normalize the pose in both position and scale.
- Returns:
A new normalized HandPose instance.
- Return type:
- normalize_position() HandPose[source]
Normalize the position of the pose (without scaling).
- Returns:
A new HandPose instance translated to a standard position.
- Return type:
- normalize_scaling() HandPose[source]
Normalize the scale of the pose (without changing position).
- Returns:
A new HandPose instance scaled to a standard size.
- Return type:
- rotate(degrees: float, axis: Literal['x', 'y', 'z'] = 'z') HandPose[source]
Rotate the pose around a specified axis.
- Parameters:
degrees (float) – The angle of rotation in degrees.
axis ({'x', 'y', 'z'}, default='z') – The axis to rotate around.
- Returns:
A new rotated HandPose instance.
- Return type:
- class handposeutils.data.handpose_sequence.HandPoseSequence(timed_poses: List[TimedHandPose] = None)[source]
Bases:
objectA sequence of hand poses with their timing information.
This class allows you to store, retrieve, and record hand poses over time, enabling playback, analysis, and live capture from a pose source.
- Parameters:
timed_poses (list of TimedHandPose, optional) – Initial list of timed hand poses. If provided, they are sorted by start_time.
- sequence
The ordered list of timed poses.
- Type:
list of TimedHandPose
- _recording_thread
The background thread for live recording.
- Type:
threading.Thread or None
- _recording
Whether a recording session is currently active.
- Type:
bool
- _record_start_time
The wall-clock time when recording started.
- Type:
float or None
See also
HandPoseHandPoseSequence is effectively a storage system for multiple HandPoses, indexed by time.
- property current_pose: HandPose | None
Get the most recent pose in the sequence.
If no time-tracking is active, this defaults to the last pose in the sequence.
- Returns:
The latest recorded pose, or None if the sequence is empty.
- Return type:
HandPose or None
- get_all_timestamps() List[float][source]
Get the start times of all poses in the sequence.
- Returns:
List of all pose start times in seconds.
- Return type:
list of float
- get_pose_at_time(timestamp: float) HandPose | None[source]
Get the hand pose active at a given timestamp.
- Parameters:
timestamp (float) – The time in seconds for which to retrieve the pose.
- Returns:
The hand pose active at the specified time, or None if no pose was active.
- Return type:
HandPose or None
- get_pose_by_index(index: int) HandPose[source]
Get a pose by its index in the sequence.
- Parameters:
index (int) – Index of the pose in the sequence.
- Returns:
The pose at the given index.
- Return type:
- Raises:
IndexError – If the index is out of range.
- start_recording(get_pose_fn: Callable[[], HandPose | None], fps: int = 30)[source]
Start recording poses from a callable source at a fixed frame rate.
- Parameters:
get_pose_fn (callable) – A function returning a HandPose or None when no pose is available.
fps (int, default=30) – Frames per second to capture.
Notes
Recording is performed in a background thread and continues until stop_recording is called.
- class handposeutils.data.handpose_sequence.TimedHandPose(pose: HandPose, start_time: float, end_time: float)[source]
Bases:
objectA hand pose with associated start and end timestamps.
- start_time
The starting time of this pose in seconds.
- Type:
float
- end_time
The ending time of this pose in seconds.
- Type:
float
- end_time: float
- start_time: float
handposeutils.embeddings package
- handposeutils.embeddings.vector.flatten_temporal_embedding(sequence: HandPoseSequence, pose_embedding_fn: Callable[[object], ndarray], max_length: int | None = 30, include_velocity: bool = True, time_scale: float = 1.0, downsample: str | None = 'uniform', pca_components: int | None = None, verbose: bool = False) ndarray[source]
Compute a flattened 1D temporal embedding vector by concatenating all frames from the structured temporal embedding.
This produces a fixed-size vector suitable for ML models requiring fixed-length input.
- Parameters:
sequence (HandPoseSequence) – Sequence of timed hand poses.
pose_embedding_fn (Callable[[HandPose], np.ndarray]) – Function to compute static embedding for each HandPose frame.
max_length (Optional[int], optional) – Target sequence length for truncation/padding (default 30).
include_velocity (bool, optional) – Whether to append velocity embeddings (default True).
time_scale (float, optional) – Frequency scale for positional encoding (default 1.0).
downsample (Optional[str], optional) – Downsampling method for longer sequences (default ‘uniform’).
pca_components (Optional[int], optional) – Dimensionality for PCA reduction (default None).
verbose (bool, optional) – If True, prints debug information (default False).
- Returns:
Flattened temporal embedding vector, where T_out is max_length or sequence length, and D_out is embedding dimension per frame after augmentation.
- Return type:
np.ndarray, shape (T_out * D_out,)
- handposeutils.embeddings.vector.get_bone_length_vector(pose: HandPose) ndarray[source]
Compute a 20-dimensional bone length vector from a HandPose.
Each element corresponds to the Euclidean length of a bone segment between two key landmarks.
Note
The HandPose should be normalized in position and scale before embedding to ensure consistent scale across samples.
- Parameters:
pose (HandPose) – Normalized hand pose from which to compute bone lengths.
- Returns:
Array of shape (20,) representing lengths of each bone segment.
- Return type:
np.ndarray
- handposeutils.embeddings.vector.get_fused_pose_embedding(pose: HandPose) ndarray[source]
Compute a 98-dimensional fused embedding vector combining: - 15D joint angles - 20D bone lengths - 63D relative landmark positions
This comprehensive vector captures intrinsic geometric and biomechanical qualities of the hand pose.
Note
Normalize the HandPose before calling this function for consistency.
- Parameters:
pose (HandPose) – Normalized hand pose to encode.
- Returns:
Concatenated embedding vector of shape (98,).
- Return type:
np.ndarray
- handposeutils.embeddings.vector.get_joint_angle_vector(pose: HandPose) ndarray[source]
Generate a 15-dimensional joint-angle embedding vector from a HandPose.
Each finger contributes three angles: - Two intra-finger joint angles between consecutive bones - One base-to-knuckle angle
The angles are measured in radians.
Note
The HandPose should be normalized in position and scale before embedding
- Parameters:
pose (HandPose) – Normalized hand pose from which to compute joint angles.
- Returns:
Array of shape (15,) containing joint angles in radians.
- Return type:
np.ndarray
- handposeutils.embeddings.vector.get_relative_vector_embedding(pose: HandPose) ndarray[source]
Compute a 63-dimensional vector of relative landmark positions to the wrist.
For each of the 21 landmarks, compute the 3D coordinate offset relative to the wrist (landmark 0), then flatten these relative coordinates into a single vector.
Note
The HandPose should be normalized before embedding for consistent comparisons.
- Parameters:
pose (HandPose) – Normalized hand pose to embed.
- Returns:
Flattened array of shape (63,), representing relative landmark positions.
- Return type:
np.ndarray
- handposeutils.embeddings.vector.structured_temporal_embedding(sequence: HandPoseSequence, pose_embedding_fn: Callable[[object], ndarray], max_length: int | None = None, include_velocity: bool = True, time_scale: float = 1.0, downsample: str | None = 'uniform', pca_components: int | None = None, verbose: bool = False) ndarray[source]
Construct a structured temporal embedding for a HandPoseSequence.
Each frame is embedded by pose_embedding_fn, augmented with sinusoidal positional encoding and optional velocity (temporal difference) embeddings. Optionally downsamples frames, applies PCA for dimensionality reduction, and pads/truncates to max_length.
- Parameters:
sequence (HandPoseSequence) – Sequence of timed hand poses.
pose_embedding_fn (Callable[[HandPose], np.ndarray]) – Function to compute static embedding for each HandPose frame.
max_length (Optional[int], optional) – Target sequence length for output embeddings. Longer sequences are truncated or downsampled. Shorter sequences are zero-padded.
include_velocity (bool, optional) – Whether to append first-order velocity embeddings (default True).
time_scale (float, optional) – Scale factor for sinusoidal time encoding frequencies (default 1.0).
downsample (Optional[str], optional) – Method to downsample frames if sequence exceeds max_length. ‘uniform’ uniformly samples frames; None disables downsampling (default ‘uniform’).
pca_components (Optional[int], optional) – If set, reduces per-frame embedding dimension to this number using PCA (default None).
verbose (bool, optional) – If True, prints debug information (default False).
- Returns:
Temporal embedding matrix with T_out = max_length (if specified) or sequence length, and D_out = per-frame embedding dimension after augmentation.
- Return type:
np.ndarray, shape (T_out, D_out)
handposeutils.visualization package
- class handposeutils.visualization.visualized_pose.VisualizedPose(coordinates, side: str = 'right_hand')[source]
Bases:
HandPoseExtended HandPose with enhanced visualization properties: visibility, opacity, annotation, finger highlighting, and color scheme.
- visible
Whether the pose is visible.
- Type:
bool
- opacity
Transparency level (0.0 = invisible, 1.0 = opaque).
- Type:
float
- annotation
Label annotation (e.g., gesture type).
- Type:
Optional[str]
- highlighted_finger
Finger name currently highlighted.
- Type:
Optional[str]
- highlight_color
RGB color for highlighted finger.
- Type:
Tuple[float, float, float]
- color_scheme
RGB colors for landmarks, finger joints, and palm.
- Type:
dict
- getColorScheme() dict[source]
Get current RGB color scheme.
- Returns:
Mapping of component names to RGB tuples.
- Return type:
dict
- highlight(finger: str, color: Tuple[float, float, float] = (1, 1, 1))[source]
Highlights a specific finger with a custom color
- setColorScheme(landmarks: Tuple[float, float, float] | None = None, fingers: list[Tuple[float, float, float]] | None = None, joints: Tuple[float, float, float] | None = None, palm: Tuple[float, float, float] | None = None)[source]
Set RGB color scheme for pose visualization components.
- Parameters:
landmarks (tuple, optional) – RGB for landmarks.
fingers (list of tuple, optional) – RGB for proximal, intermediate, distal finger joints (length 3).
joints (tuple, optional) – RGB for joints (overrides landmarks).
palm (tuple, optional) – RGB for palm.
- class handposeutils.visualization.visualizer.DeprecatedHandPoseVisualizer(window_name='Hand Pose Visualizer', color_profile: dict = None)[source]
Bases:
object- add_geometry(geometry)[source]
- Parameters:
geometry – single open3d shape to add to visulizer window
- get_landmark_point(index)[source]
- Parameters:
index – index of the landmark point to get
NOTE: will probably not work as expected if more than one hand is stored in the landmark_points list :return: landmark point at index
- read_hand_landmarks(hand, POSE_CENTER=array([0, 0, 0]))[source]
- Parameters:
landmarks – landmarks from Mediapipe input
POSE_CENTER – center of hand - EITHER np.array([x,y,z]) OR list index from landmarks of desired center [0-20]
- Returns:
None
- read_multi_landmarks(multi_hand_landmarks)[source]
- Parameters:
multi_hand_landmarks – landmarks from Mediapipe input
- Returns:
None
- set_colors(colors: dict)[source]
- Parameters:
colors – dictionary of colors for each part of the hand
default: {“landmarks”: [0.1, 0.6, 0.9], “thumb”: [0.5,0,1], “index”: [0,0.5,1], “middle”: [0,1,0.5], “ring”: [1,1,0], “pinky”: [1,0,0], “palm”: [0,1,1]} :return: None
- set_landmark_points(points_array)[source]
- Parameters:
points_array – array containing landmark points to update the screen with
- class handposeutils.visualization.visualizer.HandPoseVisualizer(window_name='Hand Pose Visualizer', color_profile: dict = None)[source]
Bases:
objectVisualizer for hand poses using Open3D.
- window_name
Name of the Open3D visualization window.
- Type:
str
- colors
Color profile for landmarks, ligaments, and palm.
- Type:
dict
- build_cached_geometry(hand_poses)[source]
Build and cache Open3D geometry for a list of hand poses.
- Parameters:
hand_poses (list) – List of HandPose instances.
- error_to_color(error, max_error)[source]
Convert an error value to an RGB color between blue (low) and red (high).
- Parameters:
error (float) – Error magnitude.
max_error (float) – Maximum error value for normalization.
- Returns:
RGB color tuple.
- Return type:
tuple
- play_sequence(hand_pose_sequence, fps=30, loop=False)[source]
Play a timed sequence of hand poses as animation.
- Parameters:
hand_pose_sequence (list) – HandPoseSequence to play.
fps (int, optional) – Frames per second playback rate.
loop (bool, optional) – Whether to loop playback indefinitely. Defaults to false.
- set_colors(colors: dict)[source]
Update color profile for visualization.
- Parameters:
colors (dict) – Colors to use for landmarks, ligaments, and palm.
- set_hand_poses(hand_pose_list)[source]
Set the list of HandPose objects to visualize.
- Parameters:
hand_pose_list (list) – List of HandPose instances.
- show_pose(finger_tips_shown=True, ligaments_shown=True, palm_shown=True)[source]
Display the current hand poses in an interactive Open3D window.
- Parameters:
finger_tips_shown (bool, optional) – Show landmark spheres.
ligaments_shown (bool, optional) – Show ligament cylinders.
palm_shown (bool, optional) – Show palm mesh.
- update_cached_geometry(hand_poses)[source]
Update cached geometry meshes with new hand pose coordinates.
- Parameters:
hand_poses (list) – List of HandPose instances.
- update_pose(finger_tips_shown=True, ligaments_shown=True, palm_shown=True)[source]
Update the visualization with the current hand poses.
- Parameters:
finger_tips_shown (bool, optional) – Show landmark spheres.
ligaments_shown (bool, optional) – Show ligament cylinders.
palm_shown (bool, optional) – Show palm mesh.