GuShH's DevBlog

Looking at the original raw images from Mars Global Surveyor among others using NASA’s image viewer (nasaview-3.18.0_win) I wondered about the file format, specifically how the data was encoded. Turns out it is a binary dump with a fixed ascii header!

I found several utilities including http://bjj.mmedia.is/utils/img2png/ (IMG2PNG from Björn Jonsson), some better than others but I couldn’t find the specifics about each format version, namely each format definition.

At first I thought the header was fixed but sometimes a field may or may not be present, at the moment I think I got most of them on the HEADER structure, at least all of the test images worked fine.

This code simply interprets the header and then reads the data. It can output the image into a BMP, PNG or JPEG.

I believe there is a bug regarding the final image dimensions, maybe someone could chip in a solution.

; NASA PDS FILE READER FOR VERSION 3
; This code is for learning purposes, it is considered incomplete and merely a starting point.
; GuShH - Gustavo Fiorenza @ 05/apr/2020
; info@gushh.net

EnableExplicit
#HEADER_MAXSIZE = 2048

Structure HEADER
	PDS_VERSION_ID.s
	FILE_NAME.s
	RECORD_TYPE.s
	RECORD_BYTES.i
	FILE_RECORDS.i
	LABEL_RECORDS.i
	IMAGE.i
	SPACECRAFT_NAME.s
	MISSION_PHASE_NAME.s
	TARGET_NAME.s
	INSTRUMENT_ID.s
	PRODUCER_ID.s
	DATA_SET_ID.s
	PRODUCT_CREATION_TIME.s
	SOFTWARE_NAME.s
	UPLOAD_ID.s
	PRODUCT_ID.s
	START_TIME.s
	IMAGE_TIME.s
	STOP_TIME.s
	SPACECRAFT_CLOCK_START_COUNT.s
	SPACECRAFT_CLOCK_STOP_COUNT.s
	FOCAL_PLANE_TEMPERATURE.s
	GAIN_MODE_ID.s
	OFFSET_MODE_ID.s
	LINE_EXPOSURE_DURATION.s
	DOWNTRACK_SUMMING.s
	CROSSTRACK_SUMMING.s
	EDIT_MODE_ID.s
	RATIONALE_DESC.s
	OBJECT.s
	LINES.i
	LINE_SAMPLES.i
	LINE_PREFIX_BYTES.i
	LINE_SUFFIX_BYTES.i
	SAMPLE_TYPE.s
	SAMPLE_BITS.i
	SAMPLE_BIT_MASK.s
	CHECKSUM.s
	END_OBJECT.s
	ENDSTR.s
EndStructure

Structure PDS
	filename.s
	filePointer.i
	header.HEADER
	readamount.i
	header_size.i
	contentDataAddress.i
	bitmap.i
	width.i
	height.i
	bits.i
	size.i
EndStructure

Procedure.s pds_readheaderparam_str( token.s )
	ProcedureReturn UCase(Trim( StringField( token, 2, "=" ) ))
EndProcedure

Procedure.i pds_readheaderparam_int( token.s )
	ProcedureReturn Val(pds_readheaderparam_str(token))
EndProcedure

Procedure.i pds_readheader( *this.PDS )
	
	Define.s header 				= ReadString(*this\filePointer, #PB_Ascii | #PB_File_IgnoreEOL)	
	Define.i lines 					= CountString( header, #LF$ )
	Define.s token 					= ""
	Define.i current_line
	
	For current_line = 1 To lines
		
		token = UCase(Trim( StringField( header, current_line, #LF$ ) ))
		token = ReplaceString( token, #CR$, #Null$, #PB_String_NoCase ) ;sanitize possible left over carriage returns from makepds version 1.3
		token = ReplaceString( token, #LF$, #Null$, #PB_String_NoCase ) ;and LF just for good measure, no need though.
		
		If token
			Debug token
			
			Select UCase(Trim( StringField( token, 1, "=" ) ))
					
				Case "PDS_VERSION_ID"
					*this\header\PDS_VERSION_ID 								= pds_readheaderparam_str( token )
				Case "FILE_NAME"
					*this\header\FILE_NAME 											= pds_readheaderparam_str( token )
				Case "RECORD_TYPE"
					*this\header\RECORD_TYPE										= pds_readheaderparam_str( token )
				Case "RECORD_BYTES"
					*this\header\RECORD_BYTES										= pds_readheaderparam_int( token )
				Case "FILE_RECORDS"
					*this\header\FILE_RECORDS										= pds_readheaderparam_int( token )
				Case "LABEL_RECORDS"
					*this\header\LABEL_RECORDS									= pds_readheaderparam_int( token )
				Case "IMAGE"
					*this\header\IMAGE													= pds_readheaderparam_int( token )
				Case "SPACECRAFT_NAME"
					*this\header\SPACECRAFT_NAME								= pds_readheaderparam_str( token )
				Case "MISSION_PHASE_NAME"
					*this\header\MISSION_PHASE_NAME							= pds_readheaderparam_str( token )
				Case "TARGET_NAME"
					*this\header\TARGET_NAME										= pds_readheaderparam_str( token )
				Case "INSTRUMENT_ID"
					*this\header\INSTRUMENT_ID									= pds_readheaderparam_str( token )
				Case "PRODUCER_ID"
					*this\header\PRODUCER_ID										= pds_readheaderparam_str( token )
				Case "DATA_SET_ID"
					*this\header\DATA_SET_ID										= pds_readheaderparam_str( token )
				Case "PRODUCT_CREATION_TIME"
					*this\header\PRODUCT_CREATION_TIME					= pds_readheaderparam_str( token )
				Case "SOFTWARE_NAME"
					*this\header\SOFTWARE_NAME									= pds_readheaderparam_str( token )
				Case "UPLOAD_ID"
					*this\header\UPLOAD_ID											= pds_readheaderparam_str( token )
				Case "PRODUCT_ID"
					*this\header\PRODUCT_ID											= pds_readheaderparam_str( token )
				Case "START_TIME"
					*this\header\START_TIME											= pds_readheaderparam_str( token )
				Case "IMAGE_TIME"
					*this\header\IMAGE_TIME											= pds_readheaderparam_str( token )
				Case "STOP_TIME"
					*this\header\STOP_TIME											= pds_readheaderparam_str( token )
				Case "SPACECRAFT_CLOCK_START_COUNT"
					*this\header\SPACECRAFT_CLOCK_START_COUNT		= pds_readheaderparam_str( token )
				Case "SPACECRAFT_CLOCK_STOP_COUNT"
					*this\header\SPACECRAFT_CLOCK_STOP_COUNT		= pds_readheaderparam_str( token )
				Case "FOCAL_PLANE_TEMPERATURE"
					*this\header\FOCAL_PLANE_TEMPERATURE				= pds_readheaderparam_str( token )
				Case "GAIN_MODE_ID"
					*this\header\GAIN_MODE_ID										= pds_readheaderparam_str( token )
				Case "OFFSET_MODE_ID"
					*this\header\OFFSET_MODE_ID									= pds_readheaderparam_str( token )
				Case "LINE_EXPOSURE_DURATION"
					*this\header\LINE_EXPOSURE_DURATION					= pds_readheaderparam_str( token )
				Case "DOWNTRACK_SUMMING"
					*this\header\DOWNTRACK_SUMMING							= pds_readheaderparam_str( token )
				Case "CROSSTRACK_SUMMING"
					*this\header\CROSSTRACK_SUMMING							= pds_readheaderparam_str( token )
				Case "EDIT_MODE_ID"
					*this\header\EDIT_MODE_ID										= pds_readheaderparam_str( token )
				Case "RATIONALE_DESC"
					*this\header\RATIONALE_DESC									= pds_readheaderparam_str( token )
				Case "OBJECT"
					*this\header\OBJECT													= pds_readheaderparam_str( token )
				Case "LINES"
					*this\header\LINES													= pds_readheaderparam_int( token )
				Case "LINE_SAMPLES"
					*this\header\LINE_SAMPLES										= pds_readheaderparam_int( token )
				Case "LINE_PREFIX_BYTES"
					*this\header\LINE_PREFIX_BYTES							= pds_readheaderparam_int( token )
				Case "LINE_SUFFIX_BYTES"
					*this\header\LINE_SUFFIX_BYTES							= pds_readheaderparam_int( token )
				Case "SAMPLE_TYPE"
					*this\header\SAMPLE_TYPE										= pds_readheaderparam_str( token )
				Case "SAMPLE_BITS"
					*this\header\SAMPLE_BITS										= pds_readheaderparam_int( token )
				Case "SAMPLE_BIT_MASK"
					*this\header\SAMPLE_BIT_MASK								= pds_readheaderparam_str( token )
				Case "CHECKSUM"
					*this\header\CHECKSUM												= pds_readheaderparam_str( token )
				Case "END_OBJECT"
					*this\header\END_OBJECT											= pds_readheaderparam_str( token )
				Case "END"
					*this\header\ENDSTR													= pds_readheaderparam_str( token )
			EndSelect
		EndIf
		
	Next
	
	; find a better way to do this!!
	FileSeek( *this\filePointer, FindString( header, "END_OBJECT = IMAGE")+Len("END_OBJECT = IMAGE") + 3 )
	
	Define.i byte = 0
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte = $0d
	
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte = $0a
	
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte <> 20 And byte <> 32
	
	ProcedureReturn #True
	
EndProcedure

Procedure.i pds_load( *this.PDS, filename.s )
	
	If *this
		If filename
			
			*this\filename 			= filename
			*this\filePointer 	= ReadFile(#PB_Any, *this\filename )
			
			If IsFile( *this\filePointer )
				If pds_readheader( *this )
					If FindString( *this\header\PDS_VERSION_ID, "PDS3" )
						
						*this\header_size 	= Loc(*this\filePointer)
						*this\width 				= *this\header\LINE_SAMPLES
						*this\height 				= *this\header\LINES
						*this\bits					= *this\header\SAMPLE_BITS
						*this\size 					= (*this\width * *this\height) * *this\bits
						
						If *this\size <= 0
							ProcedureReturn  #False
						EndIf
						
						*this\contentDataAddress = AllocateMemory( *this\size )
						*this\readamount = ReadData( *this\filePointer, *this\contentDataAddress, *this\size )
						
					Else
						Debug "error file version mismatch"
						ProcedureReturn 0
					EndIf
					
					CloseFile(*this\filePointer)
				EndIf
			EndIf
			
			ProcedureReturn *this\readamount
			
		EndIf
	EndIf
	
EndProcedure

Structure Pixel
	Pixel.l
EndStructure

Procedure.i pds_plot( *this.PDS )
	
	If *this
		If *this\header_size
			
			If *this\width > 0 And *this\height > 0
				
				*this\bitmap = CreateImage( #PB_Any, *this\width, *this\height, 32 )
				Define.byte *mem
				
				If StartDrawing( ImageOutput(*this\bitmap) )
					
					Define.i offset 			= 0
					Define.i buffer 			= DrawingBuffer()
					Define.i pitch       	= DrawingBufferPitch()
					Define.i pixelFormat 	= DrawingBufferPixelFormat()
					Define.i real_bpp 		= 0
					Define.i inversion_y 	= pixelformat ! #PB_PixelFormat_ReversedY   
					
					Select pixelformat ! #PB_PixelFormat_ReversedY
						Case #PB_PixelFormat_8Bits
							real_bpp = 1
						Case #PB_PixelFormat_15Bits
							real_bpp = 2
						Case #PB_PixelFormat_16Bits
							real_bpp = 2
						Case #PB_PixelFormat_24Bits_RGB
							real_bpp = 3
						Case #PB_PixelFormat_24Bits_BGR
							real_bpp = 3
						Case #PB_PixelFormat_32Bits_RGB
							real_bpp = 4
						Case #PB_PixelFormat_32Bits_BGR
							real_bpp = 4
					EndSelect
					
					If buffer
						
						Define.i inv_height = 0
						If inversion_y <> 0
							inv_height = *this\height
						EndIf
						
						Define.integer *buf = buffer
						Define.i x, y
						For y = 0 To *this\height - 1
							For x = 0 To *this\width - 1
								
								*mem 		= *this\contentDataAddress + *this\header_size + (( ((inv_height-y) * *this\width) + x) )
								*buf 		= buffer + (( y * pitch) + x * real_bpp)
								*buf\i 	= RGB(*mem\b, *mem\b, *mem\b )
								
							Next
						Next
					EndIf
					
					StopDrawing()
				EndIf
				
			Else
				Debug "error, image dimensions couldn't be loaded"
			EndIf
			
		EndIf
	EndIf
	
EndProcedure

Procedure.i pds_save( *this.PDS, OutputFileName.s, Format.i = #PB_ImagePlugin_BMP, Quality.i = 7 )
	
	If *this
		If Len(OutputFileName) > 0
			
			If IsImage(*this\bitmap)
				ProcedureReturn SaveImage( *this\bitmap, OutputFileName, Format, Quality )
			EndIf
			
		EndIf
	EndIf
	
EndProcedure

Procedure.i pds_create( )
	
	Define.PDS *this = AllocateMemory( SizeOf(PDS) )
	
	If *this
		InitializeStructure(*this, PDS)
		ProcedureReturn *this
	EndIf
	
EndProcedure

Procedure.i pds_destroy( *this.PDS )
	
	If *this
		If *this\contentDataAddress
			
			If IsImage(*this\bitmap)
				FreeImage(*this\bitmap)
			EndIf
			
			FreeMemory(*this\contentDataAddress)
			*this\contentDataAddress = #Null
			ClearStructure( *this, PDS )
			ProcedureReturn #True
			
		EndIf
	EndIf
	
EndProcedure

Testing images can be found at the NASA repository, a sample image is included here for studying purposes (Property of NASA).

Here is an example of how to load and convert an image:

	Define.PDS *file = pds_create()
	If *file
		pds_load( *file, "ab100402.img" )
		pds_plot( *file )
		
		If pds_save( *file, "output.bmp")
			RunProgram("output.bmp")
		EndIf
		
		pds_destroy(*file)
	EndIf

Have fun and stay safe!

This silly little tool forces harddrives to remain spinning, a few modern external disks will spin down after a certain period of inactivity. This causes all sorts of delays during access, naturally manufacturers such as Samsung won’t let you disable this option.

There are a few alternatives such as writing a batch script, python script, etc. However I like native code, so that’s what I used.

If the source is needed just ask.

Download here: KeepAlive.

Feature requests are welcomed, you can always add this to your system startup, the memory and processor requirements are minimal, enjoy: it’s free!

Cheers,
Gus

This implementation of the Marching Box Algorithm makes use of some clever programming for code readability, however it may not be inherently easy to read for beginners, but it is a good example of how to implement an algorithm such as this one, in a clean way.

It may not be the fastest implementation but this code has served me well in the past and it turns out to be quite easy to maintain as well.

The primary application is to traverse for example bitmaps, you could implement a bucket fill or follow a contour easily, however more patterns may be required depending on your requirements.

I’ve used this, among with a peucker implementation, to create rough vector representations of bitmaps in the past. You may have to add exclusions or rules if you don’t want to check the extra 4 directions in diagonal, this depends on the type of images or data you’ll be working with.

 

EnableExplicit

Enumeration
	#MARCHING_BOX_PATTERN_A
	#MARCHING_BOX_PATTERN_B
	#MARCHING_BOX_PATTERN_C
	#MARCHING_BOX_PATTERN_D
EndEnumeration

Enumeration
	#MARCHING_BOX_ADD
	#MARCHING_BOX_SUB
EndEnumeration

Structure MARCHING_BOX_PATTERN
	field.i[4] 								; holds the pattern variables.
	*variable.Integer 						; variable we'll manipulate if the pattern matches.
	operator.i 								; flat to either increment or decrement on actual *variable
EndStructure

Structure MARCHING_BOX
	List pattern.MARCHING_BOX_PATTERN()
EndStructure

Declare.i marchingbox_create( *tx.Integer, *ty.Integer, *foreground.Integer, *background.Integer )
Declare.i marchingbox_destroy( *this.MARCHING_BOX )
Declare.i marchingbox_pattern_add( *this.MARCHING_BOX, *a.Integer, *b.Integer, *c.Integer, *d.Integer, *variable.Integer, operator.i )
Declare.i marchingbox_pattern_matches( *this.MARCHING_BOX_PATTERN, a.i, b.i, c.i, d.i )

Procedure.i marchingbox_create( *tx.Integer, *ty.Integer, *foreground.Integer, *background.Integer )
	
	Define.MARCHING_BOX *this = AllocateMemory( SizeOf(MARCHING_BOX) )
	If *this
		InitializeStructure( *this, MARCHING_BOX )
		
		marchingbox_pattern_add( *this, *foreground, *foreground, *foreground, *foreground, *tx, #MARCHING_BOX_ADD )
		marchingbox_pattern_add( *this, *foreground, *background, *foreground, *foreground, *tx, #MARCHING_BOX_ADD )
		marchingbox_pattern_add( *this, *foreground, *background, *foreground, *background, *ty, #MARCHING_BOX_ADD )
		marchingbox_pattern_add( *this, *foreground, *foreground, *foreground, *background, *ty, #MARCHING_BOX_ADD )
		
		marchingbox_pattern_add( *this, *background, *background, *foreground, *foreground, *tx, #MARCHING_BOX_ADD )
		marchingbox_pattern_add( *this, *background, *foreground, *foreground, *foreground, *ty, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *background, *foreground, *background, *foreground, *ty, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *foreground, *foreground, *background, *foreground, *tx, #MARCHING_BOX_SUB )
		
		marchingbox_pattern_add( *this, *foreground, *foreground, *background, *background, *tx, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *background, *foreground, *background, *background, *ty, #MARCHING_BOX_SUB )			
		marchingbox_pattern_add( *this, *foreground, *background, *background, *background, *tx, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *background, *background, *foreground, *background, *ty, #MARCHING_BOX_ADD )
		
		marchingbox_pattern_add( *this, *background, *background, *background, *foreground, *tx, #MARCHING_BOX_ADD )
		marchingbox_pattern_add( *this, *background, *foreground, *foreground, *background, *ty, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *foreground, *background, *background, *foreground, *tx, #MARCHING_BOX_SUB )
		marchingbox_pattern_add( *this, *background, *background, *background, *background, #Null, #Null )
		
		ProcedureReturn *this
	EndIf

EndProcedure

Procedure.i marchingbox_destroy( *this.MARCHING_BOX )

	If *this
		ClearStructure( *this, MARCHING_BOX )
		ProcedureReturn FreeMemory(*this)
	EndIf

EndProcedure

Procedure.i marchingbox_pattern_add( *this.MARCHING_BOX, *a.Integer, *b.Integer, *c.Integer, *d.Integer, *variable.Integer, operator.i )
	
	If *this
		If AddElement( *this\pattern() )
			
			*this\pattern()\field[ #MARCHING_BOX_PATTERN_A ] 	= *a\i
			*this\pattern()\field[ #MARCHING_BOX_PATTERN_B ] 	= *b\i
			*this\pattern()\field[ #MARCHING_BOX_PATTERN_C ] 	= *c\i
			*this\pattern()\field[ #MARCHING_BOX_PATTERN_D ] 	= *d\i
			*this\pattern()\variable 													= *variable
			*this\pattern()\operator 													= operator
			
		EndIf
	EndIf
	
EndProcedure

Procedure.i marchingbox_pattern_matches( *this.MARCHING_BOX_PATTERN, a.i, b.i, c.i, d.i )
	
	If *this
		If *this\variable <> #Null
			If *this\field[ #MARCHING_BOX_PATTERN_A ] = a
				If *this\field[ #MARCHING_BOX_PATTERN_B ] = b
					If *this\field[ #MARCHING_BOX_PATTERN_C ] = c
						If *this\field[ #MARCHING_BOX_PATTERN_D ] = d
							
							Select *this\operator
								Case #MARCHING_BOX_ADD
									*this\variable\i + 1
								Case #MARCHING_BOX_SUB
									*this\variable\i - 1
								Default
									ProcedureReturn #False
							EndSelect
							
							ProcedureReturn #True
							
						EndIf
					EndIf
				EndIf
			EndIf
		EndIf
	EndIf
	
EndProcedure

 
Probable optimizations include denesting and less error checking on the match procedure for the release version, however that’s also implementation dependent.

This original implementation was aimed toward image manipulation, hence you’ll notice some variable names aren’t generic, but relevant to the subject. You may change this as you please.

At any rate, enjoy!

Cheers,
Gus

When having to drill repetitive holes in parts as second operations on an engine lathe, having a means to automate the tailstock could prove valuable.

Not all lathes come with the option of a hydraulic tailstock and some only have limited action.

The following are two basic ideas that can be implemented on virtually any lathe and very quickly as well!

 

First method:

Attach a suitable pneumatic cylinder to the bed using a similar clamping method as used on the tailstock itself. Have it push the lightly clamped (but free sliding) tailstock. A restrictor to control the feed-rate is mandatory. This method requires no modification to the lathe, if you need it to also retract through pneumatic action you can use a bracket or clamp on the tailstock. Otherwise you can push it back by hand as usual. Using a second restrictor on the return valve could help lengthen the life of the cylinder, so you don’t constantly ram it to fully closed all the time.

Second method:

With a long arm and bracket mounted on the toolpost, one can have the tailstock move, when the auto-feed is engaged on the carriage. This method has several limitations but it only requires light fabrication on mild-steel or aluminum for the “pushing arm”. While you can adjust the feed, it won’t be as flexible as the pneumatic cylinder, and you’ll have travel limitations in some cases.

Of course, the obvious alternative is to have the tailstock mounted in front of the carriage, and have the carriage push the lightly clamped (but still free sliding) tailstock.

 

Whichever method you choose, be careful and keep in mind possible crashes or tool ruptures. These are NON-CNC methods, for any type of electrical control you could, in theory, easily mount a stepper motor to the quill feed on the tailstock, but that itself is a modification of the lathe and beyond the scope of these two simple methods.

As for peck-drilling, the second method won’t allow it, this is why I’m starting to work on a clamp system for a pneumatic cylinder, peck-drilling with a pneumatic setup is relatively easy (although it may be expensive for some). The whole idea however is not to modify the tailstock in any way at all.

 

Enjoy!

To quickly align your tailstock:

1. Face and center-drill some scrap steel, round 1/2″ about 5 inches long.
2. Insert a dead center on the tailstock.
3. Mount your dial indicator so it touches a flat section of the dead center, right after the 60°, make sure you are on a flat by moving the tailstock quill back and forth, your indicator dial shouldn’t deviate.
4. Once you’ve setup the indicator, you can now push the dead center onto the center you drilled before. If your tailstock is misaligned your indicator will show a slight deflection as everything slightly flexes toward center.
5. At this point, you can go back and forth adjusting the tailstock setscrews until pushing the dead center won’t yield a deflection on the indicator dial.

You can easily get your tailstock centered within a thou or less using this method in under 5 minutes. Depending on whether you use a test indicator or a dial indicator, you can improve the alignment furthermore based on the available accuracy and resolution of your indicator.

Just keep in mind a test cut is always a good way to know for sure if you’re in dead center. Of course this requires the use of a known good micrometer and lots of time and patience. The alternative is to use a calibrated rod and indicate throughout the X travel for deviation, but the rod has to be held within centers.

Step 1 is very important, the part has to be center drilled on the spot every time for a good reference. You can use the same part twice (working on either side) afterwards you’ll have to face the center marks off or start from another scrap piece.

Once you are happy with the alignment:

Don’t forget to lock the setscrews against each other. Do this while indicating to make absolutely sure you aren’t pushing the tailstock to either side.

If after the alignment you still have issues, chances are you need to look into shimming your tailstock as wear has brought it’s height down from center. This is unfortunate, but it does happen. Bigger machines have removable contact plates that can be remade to deal with this, other machines can be coated or utilize teflon contact surfaces that are replacable. But typically if you have a solid cast iron base, you have to shim it.

The alternative is to machine a new lower plate / base for the tailstock, making it so the contact areas are replacable. Problem with more parts is, more chance of the parts actually moving and causing trouble. Your choice though.

This method relies on flexing of the part and the actual tailstock quill, it doesn’t matter how big or small your lathe is, all parts flex and we can use this to our advantage.