Embroidery format: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
No edit summary
Line 15: Line 15:
== Introduction ==
== Introduction ==


There are several kinds of Embroidery file formats, and each contains different abilities and features. Some formats do have different versions, with increased features. Usually these try to preserve backwards compatibility because of the cost of the hardware in question.  
There are several kinds of Embroidery file formats, and each contains different abilities and features. Some formats have different versions with increased features. Usually these try to preserve backwards compatibility because of the cost of the hardware in question.  


This page attempts to provide a short overview on embroidery formats. For technical details, see the specialized articles (menu to the right).
This page attempts to provide a short overview on embroidery formats. For technical details, see the specialized articles (menu to the right).
== Embroidery File Types ==
=== Machine Encoding ==
The most significant differences between embroidery files with the intent of running a machine, and those intended to run the programs that make those files. Often the actual controllers within the machines are similar and the encoding scheme for the embroidery files turns out to be quite similar. They are intended to control a stepper or servo motor, and a x-y plotter and a needle head. They don't do much beyond that. A lot of files simply encode for three things: a control command, an dx, and dy. This encoding is also heavily influenced by the actual physical requirement of the machine. The machines must stitch, unless the needle bar is blocked, which is the difference between a stitch and a jump. When enough of these are in a row, machines will often force a trim. It's possible with some machines to block the needle bar and move very far from the last stitch, others will force a trim. There are some cases like fringe where this is needed or puffy applique. Given this requirement, the commands used within the machine code is often similar sometimes identical when you get away from the header and into the actual stitches within the file. There are about three or four main encodings within these:
# DST encoding. Triplet code generally with the +x and +y and control bits combined in a particular fashion together. The important control bits being located on the 3rd bit with the lowest two values of the control bit always set.
# EXP encoding. 2 byte stitch, 4 byte controlled. Exp stands for Melco Expanded format so it's likely not the first to use the scheme. If the X value is set to 0x80 aka -127 this invokes control form. Where the next byte serves as a command usually with 01 being stop, 04 being jump. 0x10 being end.
# U?? encoding. Triplet code with command, x, y values as individual single bits. The first bit of the command bit is always set. The next two bits control the sign of the x and y values. Usually the last bit controls whether that command is stitched or not (jump vs. stitch).
# PEC encoding. 2, 3, 4 byte encoding. Each value of the X and Y is read individually. When the highest bit is set (0x80) it triggers long mode. It means that the top 4 bits are control (the first one being used to trigger long mode). And the bottom nibble is appended to the next byte in the stream. So if the highest bit isn't set it's a 7 bit signed number. If the highest bit is set then control bits may come into effect and the number becomes a 12 bit signed number. This allows not only trigger control events but allowing an optional long mode.
There are variations of these schemes sometimes with different endian methods and order of the bytes. Some like Xxx use 0x7F as as control, and 0x7D as long mode which means the X and Y in that case are 16 bit signed integers.
=== Vector Files ===
At the other extreme there are vector files that serve to create embroidery files by having all by storing all the data needed to create an embroidery. This means having the vector shapes and fill types, and offsets and motifs, and which order these are located the start and stop location. And then generating a bunch of line segments through a variety of algorithms. These are then set into the machine readable formats for the machines to read and follow the set commands.
=== Hybrid Machine/Vector Files ===
Some files like PES actually have both of these. They contain a pointer to a PEC block that is entirely intended for brother embroidery machines to use. These blocks also exist in PEC files that simply say #PEC0001 and then have the PEC block, or within PES files or PHB and PHC files that equally also contain a PEC block.


== Information that may be found ==
== Information that may be found ==

Revision as of 14:49, 9 July 2018

Machine embroidery
Module - entry page
Embroidery format
to finalize beginner
2018/07/09
See also

Introduction

There are several kinds of Embroidery file formats, and each contains different abilities and features. Some formats have different versions with increased features. Usually these try to preserve backwards compatibility because of the cost of the hardware in question.

This page attempts to provide a short overview on embroidery formats. For technical details, see the specialized articles (menu to the right).

Embroidery File Types

= Machine Encoding

The most significant differences between embroidery files with the intent of running a machine, and those intended to run the programs that make those files. Often the actual controllers within the machines are similar and the encoding scheme for the embroidery files turns out to be quite similar. They are intended to control a stepper or servo motor, and a x-y plotter and a needle head. They don't do much beyond that. A lot of files simply encode for three things: a control command, an dx, and dy. This encoding is also heavily influenced by the actual physical requirement of the machine. The machines must stitch, unless the needle bar is blocked, which is the difference between a stitch and a jump. When enough of these are in a row, machines will often force a trim. It's possible with some machines to block the needle bar and move very far from the last stitch, others will force a trim. There are some cases like fringe where this is needed or puffy applique. Given this requirement, the commands used within the machine code is often similar sometimes identical when you get away from the header and into the actual stitches within the file. There are about three or four main encodings within these:

  1. DST encoding. Triplet code generally with the +x and +y and control bits combined in a particular fashion together. The important control bits being located on the 3rd bit with the lowest two values of the control bit always set.
  2. EXP encoding. 2 byte stitch, 4 byte controlled. Exp stands for Melco Expanded format so it's likely not the first to use the scheme. If the X value is set to 0x80 aka -127 this invokes control form. Where the next byte serves as a command usually with 01 being stop, 04 being jump. 0x10 being end.
  3. U?? encoding. Triplet code with command, x, y values as individual single bits. The first bit of the command bit is always set. The next two bits control the sign of the x and y values. Usually the last bit controls whether that command is stitched or not (jump vs. stitch).
  4. PEC encoding. 2, 3, 4 byte encoding. Each value of the X and Y is read individually. When the highest bit is set (0x80) it triggers long mode. It means that the top 4 bits are control (the first one being used to trigger long mode). And the bottom nibble is appended to the next byte in the stream. So if the highest bit isn't set it's a 7 bit signed number. If the highest bit is set then control bits may come into effect and the number becomes a 12 bit signed number. This allows not only trigger control events but allowing an optional long mode.

There are variations of these schemes sometimes with different endian methods and order of the bytes. Some like Xxx use 0x7F as as control, and 0x7D as long mode which means the X and Y in that case are 16 bit signed integers.

Vector Files

At the other extreme there are vector files that serve to create embroidery files by having all by storing all the data needed to create an embroidery. This means having the vector shapes and fill types, and offsets and motifs, and which order these are located the start and stop location. And then generating a bunch of line segments through a variety of algorithms. These are then set into the machine readable formats for the machines to read and follow the set commands.

Hybrid Machine/Vector Files

Some files like PES actually have both of these. They contain a pointer to a PEC block that is entirely intended for brother embroidery machines to use. These blocks also exist in PEC files that simply say #PEC0001 and then have the PEC block, or within PES files or PHB and PHC files that equally also contain a PEC block.


Information that may be found

  1. Stitch Information.
    1. Direct commands go dx/dy, add stitch, go dx/dy, trim, change threads, stop.
    2. Explicit location of the points for the segment list.
    3. Stitchblocks unbroken lists of stitches in a particular color.
  2. Vector Information
    1. Shape Data, Rectangle, Circle, Path etc.
    2. How these shapes should be filled. For example:
      1. Type of fill being used
      2. Angle of the fill
      3. Angle-path of the fill
      4. Start and stop location within the shape.
      5. Pattern for the needle impacts.
      6. Randomization of edge.
  3. Font Information
    1. Text and font, how it should be applied.
  4. Design information.
    1. Design name.
    2. Design author.
    3. Design comments.
    4. Design keywords.
    5. Design copyright.
    6. Design category.
    7. Number of Stitches.
    8. Number of jumps.
    9. Size of embroidery.
    10. Start Location.
  5. Hoop Information.
    1. Specific custom hoop information.
    2. Distance design is from edge of hoop.
  6. Thread Information.
    1. Color data from a preselected list.
    2. Custom color data for thread.
    3. Thread metadata, manufacturer, pantone approximate, etc.
    4. Thread weight
  7. 2D Bitmap information, simulated view of the sewout.
    1. Bitmap representation for project. EMB contain a full color icon.
    2. Bitmap representation for each color. PEC contains 1 bit graphics.
  8. Control information for the typical editor of that format.
    1. Color of background.
    2. Scaling information.

Internal structure of some formats

  • .DST file, contains a header of 512 bytes which contains design information statistics. Then a series of direct commands. This is all that is required to run the machine.
  • .PES file, contains several different layers of information. After the header #PES00XX which determines the version of the file, it contains the position in the file containing the PEC block. The PEC block is information intended for the machine. This some information about the design, name, number of stitches, size, location of graphics information, followed by blanks equalling up to 512 bytes. And a series of direct commands for the design. This is followed by 1 bit graphic bitmaps. All of which are intended for the machine to run. Regardless of the version of the file. The program simply needs to read the location of the pec block, jump forward 512 bytes and read the direct commands. Or jump forward 22 bytes, read the graphics location, and then seek to that location and read the graphics to be displayed on the embroidery machine's screen. Different versions contain different information in the various blocks within the file that are jumped over. These include vector information and design specific instructions that are able to rebuild the stitch data from scratch. So an alteration can allow regeneration of the stitch data.
  • .EXP file, contains only direct commands.
  • .INF file, contains only color information like a thread chart.
  • .ART file, contains a Compound File Binary Format, of a series of files. They have different classes of file according to whether it contains the design information. Different files within the format contain the summary information, the Design Information, contents (the compressed stitch data, zlib 4 bytes in), the Design Icon, a bitmap of the what the design should look like. Each class denotes how much information is in the .ART file. So having the contents means it can sew, but without the design information, edits would only be possible at the stitch information. This is very similar to the .emb format and they share a common code base.

Embroidery files are used both for stitching and editing. They need to be read by the machine doing the embroidery to process the series of commands. Since most machine embroidery is rendered from shapes and fills applied to those shapes, saving only data needed to stitch would be lossy. So many formats have a hybrid of this and store easy to read stitch data and higher level objects sometimes protected with encryption and compressed (.hus, .art, .emb). With the higher level the embroidery program can reproduce the lower level stitch commands. For most programs that read this data, they often have their own higher level objects and can read only the stitch data from other formats. When they also write these formats, they very often produce the minimum acceptable version of the file that will not crash the program reading the file. So converting from Wilcom's emb to PES will produce a PES with only stitches even if the Wilcom had access to the higher level objects and the saved version of .pes also those forms available.

Kinds of available formats

  • All sorts of 2D bitmap and vector formats for the drawings, i.e. formats that are not specifically made for embroidery
  • Embroidery file formats that work on a range of machines and also can be used as exchange formats. These are sometimes called commercial formats.
  • Embroidery file formats that are mostly brand or even type specific
  • Both so-called commercial file formats and more brand-specific formats come in two forms: Some only have stitching information, others keep information that makes them easily editable. The latter could be called worker files.

It seems that there are about 30 different formats. It is not clear to me what different formats can do. Also, I don't know how formats are supported by various vendors. Some formats seem to be barely editable since they only contain stitching instructions like "go to x/y" and "add a stitch from x1/1y to x2/y2" or "change thread". Others may include precise information about the shape and kind of a design part and keep stitching information apart, i.e. an area is not just defined in terms of stitches. The latter are more easily editable. Other formats (like JEF) may keep just some information, e.g. colors.

It is sometimes difficult to find out what formats a specific machine from various Brands can read. E.g. Bernina's feature their own brand-specific editable *.art formats, but it seems that the high-end machines directly can read *.exp which is a commercial format, if I understood right. When I bought an Elna 8300, no information about formats was included in the documentation (or I couldn't find it) ...

To make the situation worse, some formats have different subtypes. E.g. the popular .PES comes in eight (actually it's no fewer than 12, but most settings in the program that edits them rather than important information) different versions :( - I once thought that the situation was really bad for video codecs, video containers or 3D vector formats, but embroidery beats anything else I am aware of in terms of obscurity and diversity.

Vendors include conversion software that can translate to their (and other's) machine readable CNC formats from a series of other low-level and also from more high-level formats. The most popular exchange formats seem to be DST and EXP, but these are not necessarily the best. As little as we know of today (after few hours spent on exploration) a good format (e.g. EMB) includes a vector description of each design part and attaches abstract stitching information to that object. This way it can be transformed without deforming stitches. Less powerful formats are directly stitch-based. The most popular rather machine-specific format seems to be *.pes (Brother) since it also includes worker information.

Embroidery file formats
extension Machine manufacturer software range Contents
ART Bernina brand-specific vectors, icon, colors, stitch
CSD Singer, POEM brand-specific
DST Tajima most programs stitch
DSG Sierra Stitch Era software worker + stitch
EMB Wilcom most high-end programs vectors, icon, colors, stitch
EXP Melco, Bernina (high-end models) most programs stitch
FDR Barudan ? ?
HUS Husqvarna Viking brand-specific, many programs stitch
JEF Janome, Elna brand-specific, many programs stitch + color
PCQ,PCD,PCM, PCS Pfaff Brand-specific stitch
PEC Bernina ? brand-specific colors, stitch, 1 bit graphics.
PES Brother popular, most programs vectors, colors, (PEC File)
SEW Janome, Elna, Kenmore most programs stitch
SHV Husqvarna Viking brand-specific stitch
VIP Pfaff (older), Husqvarna brand-specific stitch
VP3 Pfaff (newer) brand-specific stitch, color
XXX Singer, Compucon brand-specific stitch

Other lists:

Acknowledgements

Due to reorganization - i.e. the breakup of the Computerized embroidery page - names of original contributors, in particular Tatarize, do not appear in the history of this page.