                      WGT Graphics Tutorial #5
                      Topic: Realtime Morphing
                           By Chris Egerter
                            June 21, 1995

	     Contact me at: chris.egerter@homebase.com
		Compuserve: 75242,2411
                       WWW: http://kosmic.wit.com/~kosmic/gooroo/main.html

The programs in this tutorial require my graphics library, WGT 5 for
Watcom C.  You can get a shareware copy of this library from
x2ftp.oulu.fi under pub/msdos/programming/wgt/wgt51.zip.  You will also
need DOS4GW.EXE to run the compiled programs.


After writing my latest demo Flight (which can be found at kosmic.wit.com
under /kosmic/demos/flight.zip, and ftp.cdrom.com under
/pub/msdos/demos/alpha/f/flight.zip) many people have asked me how I did
the real time morphing section, so here is my response.

Morphing is well known from the Terminator 2 movie, Michael Jackson's video
Black or White, and now just about everywhere.  Morphing is the smooth change
of one object into another, and is usually done with similar objects.  You
can also pick up some publicly available image morphing programs such as
rmorph or cmorph, which will take two images and create a series of pictures
of the first picture changing into the second one.  These programs are used
for presentation graphics and the animation is drawn ahead of time.  The
difference between morphing packages and the material I'm presenting here is
my program creates each frame of the morph AS the picture is being animated.

Morphing can be broken down into two parts.  The first part is fading from
one image to another.  The second part is making the image move, or bend.

Fading
------

The image fading turned out to be the hardest part of realtime morphing.
Fading between any two 256 color pictures smoothly is not an easy task, and
takes a big precalculated table.  To keep the table small, I've limited the
pictures to 128 colors.  All pictures in the morphing sequence must share a
common palette.

To fade between any two given colors in the palette, you will need some kind
of lookup table.  You can think of the table as an array with two indices,
such as unsigned char table[128][128].  Most people would simply make an
array at the beginning of their program, but this is memory being wasted and
if you store it in a pointer, you can deallocate it halfway through your
program.  This table is used by taking any two numbers between 0 and 127,
and using them as indices in order to get back a single byte, which is a
different color.  Furthermore, since the morph will need more than one
frame of animation, we have to have one of these tables for each frame.
16 frames seemed reasonable, so I chose the table to be 16 * 128 * 128 bytes
in size.  Now you can think of the table as
unsigned char table[frame][color1][color2].  To fade between the pictures,
each frame needs a different percentage of each color.  Here is an outline
of what we need:

 Frame 1:    100.0% of color1     0.0% of color2
 Frame 2:     93.3% of color1     6.7% of color2
 Frame 3:     86.6% of color1    13.3% of color2
 Frame 4:     80.0% of color1    20.0% of color2
 Frame 5:     73.2% of color1    26.8% of color2
 Frame 6:     66.5% of color1    33.5% of color2
 Frame 7:     60.0% of color1    40.0% of color2
 Frame 8:     53.1% of color1    46.9% of color2
 Frame 9:     46.9% of color1    53.1% of color2
 Frame 10:    40.0% of color1    60.0% of color2
 Frame 11:    33.5% of color1    66.5% of color2
 Frame 12:    26.8% of color1    73.2% of color2
 Frame 13:    20.0% of color1    80.0% of color2
 Frame 14:    13.3% of color1    86.6% of color2
 Frame 15:     6.7% of color1    93.3% of color2
 Frame 16:     0.0% of color1   100.0% of color2

(Note these numbers are not accurate because I did them by hand on a
 calculator.  It doesn't matter because the program calculates them.)

Notice that as the percentage of the first picture decreases, the percentage
of the second picture increases by the same amount.

The percentage of color2 is always equal to 100% - color1.

We need an algorithm that will return a new color given two other colors
and their weighted value.  To do this, we have to work with the red, green,
and blue values of the colors.

For example, if color1 is (64, 64, 64) and color2 is (0, 0, 0) and we want
to take 80% of color1 and 20% of color2:

0.8 * 64 = 51.2
0.2 *  0 = 0

From this we construct a new color with RGB components by adding the two
weighted values together.

R = 0.8 * 64 + 0.2 * 0 = 51.2
G = 0.8 * 64 + 0.2 * 0 = 51.2
B = 0.8 * 64 + 0.2 * 0 = 51.2

The formula then is:

R = percentage_of_color1 * color1_Red   + percentage_of_color2 * color2_Red
G = percentage_of_color1 * color1_Green + percentage_of_color2 * color2_Green
B = percentage_of_color1 * color1_Blue  + percentage_of_color2 * color2_Blue

This will give us a totally new color which may or may not be defined in the
palette we are using.  The trick is to find out which color matches the
closest.  We will use color quantization in order to find the closest match.

This algorithm uses the general distance formula:

D = sqrt (delta_R ^ 2 + delta_G ^ 2 + delta_B ^ 2)

After we calculate the desired RGB values, this formala is applied to each
existing color in the palette.  The color with the shortest distance will
be the closest match.  The delta values are the difference between the
new color and the existing color.


Here is the routine which finds the closest match given some RGB values:
Note I have defined MAXCOL as 128 for this example.

unsigned char closest_match (int red, int green, int blue, color *pal)
{
int colnum;                     /* Color number to compare */
int delta_r, delta_g, delta_b;  /* Difference between new and existing */
float distance;                 /* Distance from new color */
float lowest_distance;          /* Lowest distance found */
int closest_color;              /* Color with the lowest distance */


   lowest_distance = 3333;
   /* Set to a high number, higher than anything possible.
      The highest distance we can have is sqrt (64*64 + 64*64 + 64*64)
      which is 110.85 */

   for  (colnum = 0; colnum < MAXCOL; colnum++)
   /* Compare each color from the existing palette */
     {

      delta_r = (pal[colnum].r - red);
      delta_g = (pal[colnum].g - green);
      delta_b = (pal[colnum].b - blue);

      distance = sqrt (delta_r*delta_r + delta_g*delta_g + delta_b*delta_b);

      if  (distance < lowest_distance)
        {
         lowest_distance = distance;
         closest_color = colnum;
        }
     }

 return (closest_color);
}



The next routine loops through all the colors in the palette and stores
the closest matches into the table, given a frame and number of frames.

void create_fade_table (int frame, int maxframe,
                        unsigned char *fadetable, color *pal)
{

float lightlevel1;      /* Percentage of color 1 */
float lightlevel2;      /* Percentage of color 2 */

float fr, fg, fb;       /* Floating point RGB values of color 1 */
float fr2, fg2, fb2;    /* Floating point RGB values of color 2 */
int   ir,  ig,  ib;     /* Integer RGB values after combing the two above */

int col1, col2;

unsigned char bestcolor; /* Best match */

 lightlevel1 = (float)frame / (float)maxframe;
 /* Calculate the percentage of color 1 */

 lightlevel2 = 1.0 - lightlevel1;
 /* Percentage of color 2 is the 100% - lightlevel1 */


 for (col2 = 0; col2 < MAXCOL; col2++)
   for (col = 0; col < MAXCOL; col++)
     {
      
      fr = (float)pal[col].r * lightlevel1;
      fg = (float)pal[col].g * lightlevel1;
      fb = (float)pal[col].b * lightlevel1;

      fr2= (float)pal[col2].r * lightlevel2;
      fg2= (float)pal[col2].g * lightlevel2;
      fb2= (float)pal[col2].b * lightlevel2;

      /* Calculate the two new colors */

      ir = (fr + fr2);
      ig = (fg + fg2);
      ib = (fb + fb2);
      /* Combine the percentage of color 1 with the percentage of color 2
         to form a new color */

      bestmatch = closest_match (ir, ig, ib, pal);

      fadetable[frame * (MAXCOL*MAXCOL) + col2 * MAXCOL + col] = bestmatch;
      /* Store the color at fadetable[frame][col2][col1] */  

  }

}



We also need a routine to calculate the data for each frame:


void create_all_frames (int numframes, unsigned char *fadetable,
                        color *pal)
{
int i;

 for (i = 0; i <= numframes; i++)
  { 
   printf ("Calculating Frame: %hi\n", i);
   create_fade_table (i, numframes, fadetable, pal);
  }
}



That was all nice and dandy, but how are we going to USE this table??!

We have pointers to the two pictures, a big table, and a pointer to a
destination where the result of the fade will go.

For my demo, I used 256x200 pictures.  The width is limited to 256 because
of the texture mapping routines.

Now to create a single frame of the fade, we will use the following method:

For every pixel
 {
  Take a pixel from picture 1
  Take a pixel from picture 2
  Combine these values into a single number by
   newvalue = pixel1 * MAXCOL + pixel2
  Get a new color from the table at offset newvalue
  Stored the new color in the destination
  Move to the next pixel in picture 1, picture 2, and destination
 }

Since we're dealing with all these operations for every pixel, we're going
to need some assembly language for speed.

 esi = picture 1
 edi = picture 2
 ebp = fade table
 eax = destination
 ebx = offset into ebp

For a single pixel, do the following:

 mov bl, 0              /* Clear out the lower byte of BX */
 mov bh, [esi]          /* Get a byte from picture 1 */
 shr bx, 1              /* Shift right, so BX now contains color1 * 128 */
 add bl, [edi]          /* Add a byte from picture 2
                           BX now contains color1 * 128 + color2 */
 mov dl, [ebp + ebx]    /* Get a byte from the table and store it in DL */
 inc esi                /* Move to the next pixel in picture 1 and 2 */
 inc edi

Note I stored the new color in dl, but haven't stored it into the
destination yet.  We can move two bytes at a time if we repeat the above
code and store the second byte in dh.

*CAUTION*  Before using ebx as an offset, you first need to clear it out.
Otherwise the high word of ebx may contain a value and the offset will be
outside the fade table.

Watcom's inline assembly doesn't seem to allow the use of EBP as a parameter,
so I pushed and poped the value instead.

Here is the full inline assembly routine which creates a frame of the fade.

void fade_frame_asm (block pic1, block pic2, block table, block dest,
                     int length);
#pragma aux fade_frame_asm = \
 "push ebp" \
 "mov ebp, edx" \
 "mov ebx, 0" \
 "fadeloop: mov bl, 0" \
 "mov bh, [esi]" \
 "shr bx, 1" \
 "add bl, [edi]" \
 "mov dl, [ebp + ebx]" \
 "inc esi" \
 "inc edi" \
 "mov bl, 0" \
 "mov bh, [esi]" \
 "shr bx, 1" \
 "add bl, [edi]" \
 "mov dh, [ebp + ebx]" \
 "inc esi" \
 "inc edi" \
 "mov [eax], dx" \
 "add eax, 2" \
 "dec ecx" \
 "jnz fadeloop" \
 "pop ebp"\
parm [esi] [edi] [edx] [eax] [ecx] \
modify exact [eax ebx ecx edx esi edi] nomemory;


Before using this routine, we need to adjust the pointers because WGT stores
the width and height before every block.  As well, the fade table pointer
must be adjust depending on which frame is being shown.

void fade_frame (block pic1, block pic2, block dest, block fade_table,
                 int frame)
{
 fade_table += frame * (MAXCOL * MAXCOL);
 pic1 += 4;
 pic2 += 4;
 dest += 4;

 fade_frame_asm (pic1, pic2, fade_table, dest, 25600);
 /* Length is 256*200 / 2 because we're moving words */
}


Let's put all of this together to make a program which fades between two
pictures.

Below is the listing for fade.c:
----------------------------------FADE.C--------------------------------------
#include <wgt5.h>
#include <stdio.h>
#include <dos.h>
#include <math.h>
#include <malloc.h>

#define MAXCOL 128L
#define NUMFRAMES 16L

int oldmode;

block picture1;
block picture2;
block destination;
color pal[256];

unsigned char *fade_table;



/* Fade table creation routines */
/* ------------------------------------------------------------------------ */
unsigned char closest_match (int red, int green, int blue, color *pal)
{
int colnum;                     /* Color number to compare */
int delta_r, delta_g, delta_b;  /* Difference between new and existing */
float distance;                 /* Distance from new color */
float lowest_distance;          /* Lowest distance found */
int closest_color;              /* Color with the lowest distance */


   lowest_distance = 33333;
   /* Set to a high number, higher than anything possible.
      The highest distance we can have is sqrt (64*64 + 64*64 + 64*64)
      which is 110.85 */

   for  (colnum = 0; colnum < MAXCOL; colnum++)
   /* Compare each color from the existing palette */
     {

      delta_r = (pal[colnum].r - red);
      delta_g = (pal[colnum].g - green);
      delta_b = (pal[colnum].b - blue);

      distance = sqrt (delta_r*delta_r*30 + delta_g*delta_g*59 + delta_b*delta_b*11);

      if  (distance < lowest_distance)
        {
         lowest_distance = distance;
         closest_color = colnum;
        }
     }

 return (closest_color);
}



void create_fade_table (int frame, int maxframe,
                        unsigned char *fadetable, color *pal)
{

float lightlevel1;      /* Percentage of color 1 */
float lightlevel2;      /* Percentage of color 2 */

float fr, fg, fb;       /* Floating point RGB values of color 1 */
float fr2, fg2, fb2;    /* Floating point RGB values of color 2 */
int   ir,  ig,  ib;     /* Integer RGB values after combing the two above */

int col1, col2;

unsigned char bestcolor; /* Best match */

 lightlevel1 = (float)frame / (float)maxframe;
 /* Calculate the percentage of color 1 */

 lightlevel2 = 1.0 - lightlevel1;
 /* Percentage of color 2 is the 100% - lightlevel1 */


 for (col2 = 0; col2 < MAXCOL; col2++)
   for (col1 = 0; col1 < MAXCOL; col1++)
     {
      
      fr = (float)pal[col1].r * lightlevel1;
      fg = (float)pal[col1].g * lightlevel1;
      fb = (float)pal[col1].b * lightlevel1;

      fr2= (float)pal[col2].r * lightlevel2;
      fg2= (float)pal[col2].g * lightlevel2;
      fb2= (float)pal[col2].b * lightlevel2;

      /* Calculate the two new colors */

      ir = (fr + fr2);
      ig = (fg + fg2);
      ib = (fb + fb2);
      /* Combine the percentage of color 1 with the percentage of color 2
         to form a new color */

      bestcolor = closest_match (ir, ig, ib, pal);

      fadetable[frame * (MAXCOL*MAXCOL) + col2 * MAXCOL + col1] = bestcolor;
      wsetcolor (bestcolor);
      wputpixel (col1, col2);
      /* Store the color at fadetable[frame][col2][col1] */

  }

}



void create_all_frames (int numframes, unsigned char *fadetable,
                        color *pal)
{
int i;

 wtextcolor (255);  /* White color */
 wtexttransparent (TEXTFGBG);

 for (i = 0; i < numframes; i++)
  { 
   wgtprintf (160, 0, NULL, "Frame: %hi/%hi", i, numframes-1);
   create_fade_table (i, numframes, fadetable, pal);
  }
}




/* Frame creation routines */
/* ------------------------------------------------------------------------ */

void fade_frame_asm (block pic1, block pic2, block table, block dest, int length);
#pragma aux fade_frame_asm = \
 "push ebp" \
 "mov ebp, edx" \
 "mov ebx, 0" \
 "fadeloop: mov bl, 0" \
 "mov bh, [esi]" \
 "shr bx, 1" \
 "add bl, [edi]" \
 "mov dl, [ebp + ebx]" \
 "inc esi" \
 "inc edi" \
 "mov bl, 0" \
 "mov bh, [esi]" \
 "shr bx, 1" \
 "add bl, [edi]" \
 "mov dh, [ebp + ebx]" \
 "inc esi" \
 "inc edi" \
 "mov [eax], dx" \
 "add eax, 2" \
 "dec ecx" \
 "jnz fadeloop" \
 "pop ebp"\
parm [esi] [edi] [edx] [eax] [ecx] \
modify exact [eax ebx ecx edx esi edi] nomemory;


void fade_frame (block pic1, block pic2, block dest, block fade_table,
                 int frame)
{
 fade_table += frame * (MAXCOL * MAXCOL);
 pic1 += 4;
 pic2 += 4;
 dest += 4;

 fade_frame_asm (pic1, pic2, fade_table, dest, 25600);
 /* Length is 256*200 / 2 because we're moving words */
}



void save_table (char *filename, block table, long size)
{
FILE *out;
int i;

 out = fopen (filename, "wb");
 fwrite (table, size, 1, out);
 fclose (out);
}


block load_table (char *filename, long size)
{
FILE *in;
int i;
block table;

 if  ((in = fopen (filename, "rb")) == NULL)
     return NULL;

 table = (unsigned char *)malloc (size);
 fread (table, size, 1, libf);
 fclose (libf);
 return table;
}


void main (void)
{
int frame;
int framedir;

 oldmode = wgetmode ();

 vga256 ();

 picture1 = wloadpcx ("amber.pcx", pal);
 picture2 = wloadpcx ("termnate.pcx", pal);
 destination = wallocblock (256, 200);

 wsetrgb (255, 63, 63, 63, pal);
 /* Create a white color for text */

 wsetpalette (0, 255, pal);


 fade_table = load_table ("fade.tab", MAXCOL * MAXCOL * NUMFRAMES);
 if (fade_table == NULL)
   {

    wresize (0, 140, 50, 190, picture1, 0);
    wresize (100, 140, 150, 190, picture2, 0);
    /* Show some thumbnail versions of the pictures to fade */

    fade_table = malloc (MAXCOL * MAXCOL * NUMFRAMES);
    create_all_frames (NUMFRAMES, fade_table, pal);
    save_table ("fade.tab", fade_table, MAXCOL * MAXCOL * NUMFRAMES);
   }


 wcls (0);

 framedir = 1;
 frame = 0;

 do
   {
    fade_frame (picture1, picture2, destination, fade_table, frame);
    wputblock (32, 0, destination, 0);

    frame += framedir;
    if (frame == NUMFRAMES - 1)
      framedir = -1;
    if (frame == 0)
      framedir = 1;
   } while (!kbhit ());     


 free (fade_table);
 wfreeblock (picture1);
 wfreeblock (picture2);
 wfreeblock (destination);
 wsetmode (oldmode);
}





The first time you run this program, it will create the fade table called
fade.tab and save it to disk.  If the table already exists, it loads it
instead of doing all the calculations again.  During the table creation
process, it will display a 128x128 array of the colors.  You will notice
that the table is symmetrical.

Take another look at the values I've listed beforehand.

 Frame 8:     53.1% of color1    46.9% of color2
 Frame 9:     46.9% of color1    53.1% of color2

Notice the percentages are swapped between these frames.  This means the
table is going to have duplicate values!  Therefore we can cut the memory
required for the table in half, simply by swapping picture1 and picture2
if the frame is greater than half the total number of frames.

    if (frame < NUMFRAMES / 2)
      fade_frame (picture1, picture2, destination, fade_table, frame);
    else
      fade_frame (picture2, picture1, destination, fade_table,
                  (NUMFRAMES - 1) - frame);

This simple modification cuts the table size in half, and produces exactly
the same result.

Try running fade2.exe, which creates fade2.tab.  This time the table is
half as big, so it doesn't take as long to create it.



Meshes
------
The second part of the morphing process is the texture mapped triangular
mesh.  If you've tried any of the morphing creation software, you will know
that a grid of points is created and curved lines are drawn between them.
These points move from one position to another during the morph.  Since
curved lines would require too many calculations to do realtime, I've used
plain triangles.  The first thing I did was create a grid of points overtop
each picture.  For the demo I used a grid of 6x6 vertices.  The next step
was to make a mesh of triangles based on these vertices.

Below is a nice ASCII diagram of how the mesh looks.  Each vertex is marked
with an asterisk.

*-----*-----*-----*-----*-----*
| \   |   / | \   |   / | \   |
|   \ | /   |   \ | /   |   \ |
*-----*-----*-----*-----*-----*
|   / | \   |   / | \   |   / |
| /   |   \ | /   |   \ | /   |
*-----*-----*-----*-----*-----*
| \   |   / | \   |   / | \   |
|   \ | /   |   \ | /   |   \ |
*-----*-----*-----*-----*-----*
|   / | \   |   / | \   |   / |
| /   |   \ | /   |   \ | /   |
*-----*-----*-----*-----*-----*
| \   |   / | \   |   / | \   |
|   \ | /   |   \ | /   |   \ |
*-----*-----*-----*-----*-----*

We really don't have to worry about each triangle.  This mesh is created FROM
the vertices, so if the middle vertex moves, all the triangles sharing
the vertex change.   Triangles DO NOT slide around on the screen.  They can
only stretch to a different size.  When moving the vertices, the resulting
triangle mesh should never overlap, or the morph will be drawn incorrectly.

A separate mesh is stored for each picture.  This enables us to assign the
vertices to certain reference spots on each picture.  Say you had two
different pictures of faces.  Faces are good for morphing because they have
several reference spots that are similar, such as the pupils, chin, nose,
hairline, ears, etc.  I usually assign the first vertices to the eyes,
because they are the most prominent feature.  To set up the mesh, you need
to pick a reference spot on the both images, find a close vertex on the first
mesh, and move the vertex on top of the reference spot.  You also have to
move the SAME vertex on the second mesh to the reference spot on the second
picture.  The whole idea behind morphing is that these reference spots are
going to be in different places on each picture.


To move each vertex, I have used fixed point.    The speed of each vertex
is stored in an array called dirs, and the values are calculated by taking
the difference between the vertices in each mesh, and dividing it by the
number of frames in the sequence.  

  dirs[y][x].x = ((mesh2[y][x].x - mesh1[y][x].x) * 256) / NUMFRAMES;
  dirs[y][x].y = ((mesh2[y][x].y - mesh1[y][x].y) * 256) / NUMFRAMES;

When each frame is drawn, the vertices have this amount added to them, which
makes them move from one spot to another.


HOW IT WORKS
------------

The vertices start out as the first mesh, and move to the positions in the
second mesh.  The resulting triangles are texture mapped from the pictures,
and drawn in another buffer.  The texture coordinates remain the same
throughout the sequence, but the screen coordinates move with the vertices.
Both meshes are texture mapped at the same time, onto different buffers.
Each buffer will be used as source images for our fade routine.  All that is
left is to fade the two images together and show the result on the screen.

The vertex positions for each picture are the same, but the texture is
different.  This means that when the images are fading, both images are
stretching in the same direction.  The result is a smooth morph from one
picture to another.



MORPH.EXE
---------
In the morph.exe program, I have designed a simple way to set up your
mesh.  It displays two pictures with a mesh over them, and lets you click
on a vertex.  When you click again, the vertex moves to a new location.
You can test the morph sequence at any time by pressing M.  To load and
save your mesh, press L or S.  The will automatically use the files mesh1.dat
and mesh2.dat.  If you want to use the mesh you've create later, copy them
to a unique filename.  Pressing Q will exit the program.


DEMO.EXE
--------
This program reads the meshes from demo1.dat and demo2.dat and alternates
between the two images.  You can also modify this program to chain several
sequences together, to make multiple objects morph into each other.


Last Note:  To make your own pictures, make sure they only use the first
128 colors, and share the same palette.
