In the Windows XP there is a little avi video called "clock.avi" which you could use as an example.
When you compile, you will see the avi embedded in your help file. This is contained within your .CHM file so you don't need to ship it. If your avi's are very big and you want to ship then separately then replace the above html with:
<embed src="type your full path/clock.avi">
The disadvantage is that you need the full path and you need to ship the avi files with your .chm file.