Category: Miscellaneous

Reading NASA’s PDS files for conversion purposes

Posted by on April 5, 2020

Looking at the original raw images from Mars Global Surveyor among others using NASA’s image viewer (nasaview-3.18.0_win) I wondered about the file format, specifically how the data was encoded. Turns out it is a binary dump with a fixed ascii header!

I found several utilities including http://bjj.mmedia.is/utils/img2png/ (IMG2PNG from Björn Jonsson), some better than others but I couldn’t find the specifics about each format version, namely each format definition.

At first I thought the header was fixed but sometimes a field may or may not be present, at the moment I think I got most of them on the HEADER structure, at least all of the test images worked fine.

This code simply interprets the header and then reads the data. It can output the image into a BMP, PNG or JPEG.

I believe there is a bug regarding the final image dimensions, maybe someone could chip in a solution.

; NASA PDS FILE READER FOR VERSION 3
; This code is for learning purposes, it is considered incomplete and merely a starting point.
; GuShH - Gustavo Fiorenza @ 05/apr/2020
; info@gushh.net

EnableExplicit
#HEADER_MAXSIZE = 2048

Structure HEADER
	PDS_VERSION_ID.s
	FILE_NAME.s
	RECORD_TYPE.s
	RECORD_BYTES.i
	FILE_RECORDS.i
	LABEL_RECORDS.i
	IMAGE.i
	SPACECRAFT_NAME.s
	MISSION_PHASE_NAME.s
	TARGET_NAME.s
	INSTRUMENT_ID.s
	PRODUCER_ID.s
	DATA_SET_ID.s
	PRODUCT_CREATION_TIME.s
	SOFTWARE_NAME.s
	UPLOAD_ID.s
	PRODUCT_ID.s
	START_TIME.s
	IMAGE_TIME.s
	STOP_TIME.s
	SPACECRAFT_CLOCK_START_COUNT.s
	SPACECRAFT_CLOCK_STOP_COUNT.s
	FOCAL_PLANE_TEMPERATURE.s
	GAIN_MODE_ID.s
	OFFSET_MODE_ID.s
	LINE_EXPOSURE_DURATION.s
	DOWNTRACK_SUMMING.s
	CROSSTRACK_SUMMING.s
	EDIT_MODE_ID.s
	RATIONALE_DESC.s
	OBJECT.s
	LINES.i
	LINE_SAMPLES.i
	LINE_PREFIX_BYTES.i
	LINE_SUFFIX_BYTES.i
	SAMPLE_TYPE.s
	SAMPLE_BITS.i
	SAMPLE_BIT_MASK.s
	CHECKSUM.s
	END_OBJECT.s
	ENDSTR.s
EndStructure

Structure PDS
	filename.s
	filePointer.i
	header.HEADER
	readamount.i
	header_size.i
	contentDataAddress.i
	bitmap.i
	width.i
	height.i
	bits.i
	size.i
EndStructure

Procedure.s pds_readheaderparam_str( token.s )
	ProcedureReturn UCase(Trim( StringField( token, 2, "=" ) ))
EndProcedure

Procedure.i pds_readheaderparam_int( token.s )
	ProcedureReturn Val(pds_readheaderparam_str(token))
EndProcedure

Procedure.i pds_readheader( *this.PDS )
	
	Define.s header 				= ReadString(*this\filePointer, #PB_Ascii | #PB_File_IgnoreEOL)	
	Define.i lines 					= CountString( header, #LF$ )
	Define.s token 					= ""
	Define.i current_line
	
	For current_line = 1 To lines
		
		token = UCase(Trim( StringField( header, current_line, #LF$ ) ))
		token = ReplaceString( token, #CR$, #Null$, #PB_String_NoCase ) ;sanitize possible left over carriage returns from makepds version 1.3
		token = ReplaceString( token, #LF$, #Null$, #PB_String_NoCase ) ;and LF just for good measure, no need though.
		
		If token
			Debug token
			
			Select UCase(Trim( StringField( token, 1, "=" ) ))
					
				Case "PDS_VERSION_ID"
					*this\header\PDS_VERSION_ID 								= pds_readheaderparam_str( token )
				Case "FILE_NAME"
					*this\header\FILE_NAME 											= pds_readheaderparam_str( token )
				Case "RECORD_TYPE"
					*this\header\RECORD_TYPE										= pds_readheaderparam_str( token )
				Case "RECORD_BYTES"
					*this\header\RECORD_BYTES										= pds_readheaderparam_int( token )
				Case "FILE_RECORDS"
					*this\header\FILE_RECORDS										= pds_readheaderparam_int( token )
				Case "LABEL_RECORDS"
					*this\header\LABEL_RECORDS									= pds_readheaderparam_int( token )
				Case "IMAGE"
					*this\header\IMAGE													= pds_readheaderparam_int( token )
				Case "SPACECRAFT_NAME"
					*this\header\SPACECRAFT_NAME								= pds_readheaderparam_str( token )
				Case "MISSION_PHASE_NAME"
					*this\header\MISSION_PHASE_NAME							= pds_readheaderparam_str( token )
				Case "TARGET_NAME"
					*this\header\TARGET_NAME										= pds_readheaderparam_str( token )
				Case "INSTRUMENT_ID"
					*this\header\INSTRUMENT_ID									= pds_readheaderparam_str( token )
				Case "PRODUCER_ID"
					*this\header\PRODUCER_ID										= pds_readheaderparam_str( token )
				Case "DATA_SET_ID"
					*this\header\DATA_SET_ID										= pds_readheaderparam_str( token )
				Case "PRODUCT_CREATION_TIME"
					*this\header\PRODUCT_CREATION_TIME					= pds_readheaderparam_str( token )
				Case "SOFTWARE_NAME"
					*this\header\SOFTWARE_NAME									= pds_readheaderparam_str( token )
				Case "UPLOAD_ID"
					*this\header\UPLOAD_ID											= pds_readheaderparam_str( token )
				Case "PRODUCT_ID"
					*this\header\PRODUCT_ID											= pds_readheaderparam_str( token )
				Case "START_TIME"
					*this\header\START_TIME											= pds_readheaderparam_str( token )
				Case "IMAGE_TIME"
					*this\header\IMAGE_TIME											= pds_readheaderparam_str( token )
				Case "STOP_TIME"
					*this\header\STOP_TIME											= pds_readheaderparam_str( token )
				Case "SPACECRAFT_CLOCK_START_COUNT"
					*this\header\SPACECRAFT_CLOCK_START_COUNT		= pds_readheaderparam_str( token )
				Case "SPACECRAFT_CLOCK_STOP_COUNT"
					*this\header\SPACECRAFT_CLOCK_STOP_COUNT		= pds_readheaderparam_str( token )
				Case "FOCAL_PLANE_TEMPERATURE"
					*this\header\FOCAL_PLANE_TEMPERATURE				= pds_readheaderparam_str( token )
				Case "GAIN_MODE_ID"
					*this\header\GAIN_MODE_ID										= pds_readheaderparam_str( token )
				Case "OFFSET_MODE_ID"
					*this\header\OFFSET_MODE_ID									= pds_readheaderparam_str( token )
				Case "LINE_EXPOSURE_DURATION"
					*this\header\LINE_EXPOSURE_DURATION					= pds_readheaderparam_str( token )
				Case "DOWNTRACK_SUMMING"
					*this\header\DOWNTRACK_SUMMING							= pds_readheaderparam_str( token )
				Case "CROSSTRACK_SUMMING"
					*this\header\CROSSTRACK_SUMMING							= pds_readheaderparam_str( token )
				Case "EDIT_MODE_ID"
					*this\header\EDIT_MODE_ID										= pds_readheaderparam_str( token )
				Case "RATIONALE_DESC"
					*this\header\RATIONALE_DESC									= pds_readheaderparam_str( token )
				Case "OBJECT"
					*this\header\OBJECT													= pds_readheaderparam_str( token )
				Case "LINES"
					*this\header\LINES													= pds_readheaderparam_int( token )
				Case "LINE_SAMPLES"
					*this\header\LINE_SAMPLES										= pds_readheaderparam_int( token )
				Case "LINE_PREFIX_BYTES"
					*this\header\LINE_PREFIX_BYTES							= pds_readheaderparam_int( token )
				Case "LINE_SUFFIX_BYTES"
					*this\header\LINE_SUFFIX_BYTES							= pds_readheaderparam_int( token )
				Case "SAMPLE_TYPE"
					*this\header\SAMPLE_TYPE										= pds_readheaderparam_str( token )
				Case "SAMPLE_BITS"
					*this\header\SAMPLE_BITS										= pds_readheaderparam_int( token )
				Case "SAMPLE_BIT_MASK"
					*this\header\SAMPLE_BIT_MASK								= pds_readheaderparam_str( token )
				Case "CHECKSUM"
					*this\header\CHECKSUM												= pds_readheaderparam_str( token )
				Case "END_OBJECT"
					*this\header\END_OBJECT											= pds_readheaderparam_str( token )
				Case "END"
					*this\header\ENDSTR													= pds_readheaderparam_str( token )
			EndSelect
		EndIf
		
	Next
	
	; find a better way to do this!!
	FileSeek( *this\filePointer, FindString( header, "END_OBJECT = IMAGE")+Len("END_OBJECT = IMAGE") + 3 )
	
	Define.i byte = 0
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte = $0d
	
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte = $0a
	
	Repeat
		byte = ReadByte(*this\filePointer)
	Until byte <> 20 And byte <> 32
	
	ProcedureReturn #True
	
EndProcedure

Procedure.i pds_load( *this.PDS, filename.s )
	
	If *this
		If filename
			
			*this\filename 			= filename
			*this\filePointer 	= ReadFile(#PB_Any, *this\filename )
			
			If IsFile( *this\filePointer )
				If pds_readheader( *this )
					If FindString( *this\header\PDS_VERSION_ID, "PDS3" )
						
						*this\header_size 	= Loc(*this\filePointer)
						*this\width 				= *this\header\LINE_SAMPLES
						*this\height 				= *this\header\LINES
						*this\bits					= *this\header\SAMPLE_BITS
						*this\size 					= (*this\width * *this\height) * *this\bits
						
						If *this\size <= 0
							ProcedureReturn  #False
						EndIf
						
						*this\contentDataAddress = AllocateMemory( *this\size )
						*this\readamount = ReadData( *this\filePointer, *this\contentDataAddress, *this\size )
						
					Else
						Debug "error file version mismatch"
						ProcedureReturn 0
					EndIf
					
					CloseFile(*this\filePointer)
				EndIf
			EndIf
			
			ProcedureReturn *this\readamount
			
		EndIf
	EndIf
	
EndProcedure

Structure Pixel
	Pixel.l
EndStructure

Procedure.i pds_plot( *this.PDS )
	
	If *this
		If *this\header_size
			
			If *this\width > 0 And *this\height > 0
				
				*this\bitmap = CreateImage( #PB_Any, *this\width, *this\height, 32 )
				Define.byte *mem
				
				If StartDrawing( ImageOutput(*this\bitmap) )
					
					Define.i offset 			= 0
					Define.i buffer 			= DrawingBuffer()
					Define.i pitch       	= DrawingBufferPitch()
					Define.i pixelFormat 	= DrawingBufferPixelFormat()
					Define.i real_bpp 		= 0
					Define.i inversion_y 	= pixelformat ! #PB_PixelFormat_ReversedY   
					
					Select pixelformat ! #PB_PixelFormat_ReversedY
						Case #PB_PixelFormat_8Bits
							real_bpp = 1
						Case #PB_PixelFormat_15Bits
							real_bpp = 2
						Case #PB_PixelFormat_16Bits
							real_bpp = 2
						Case #PB_PixelFormat_24Bits_RGB
							real_bpp = 3
						Case #PB_PixelFormat_24Bits_BGR
							real_bpp = 3
						Case #PB_PixelFormat_32Bits_RGB
							real_bpp = 4
						Case #PB_PixelFormat_32Bits_BGR
							real_bpp = 4
					EndSelect
					
					If buffer
						
						Define.i inv_height = 0
						If inversion_y <> 0
							inv_height = *this\height
						EndIf
						
						Define.integer *buf = buffer
						Define.i x, y
						For y = 0 To *this\height - 1
							For x = 0 To *this\width - 1
								
								*mem 		= *this\contentDataAddress + *this\header_size + (( ((inv_height-y) * *this\width) + x) )
								*buf 		= buffer + (( y * pitch) + x * real_bpp)
								*buf\i 	= RGB(*mem\b, *mem\b, *mem\b )
								
							Next
						Next
					EndIf
					
					StopDrawing()
				EndIf
				
			Else
				Debug "error, image dimensions couldn't be loaded"
			EndIf
			
		EndIf
	EndIf
	
EndProcedure

Procedure.i pds_save( *this.PDS, OutputFileName.s, Format.i = #PB_ImagePlugin_BMP, Quality.i = 7 )
	
	If *this
		If Len(OutputFileName) > 0
			
			If IsImage(*this\bitmap)
				ProcedureReturn SaveImage( *this\bitmap, OutputFileName, Format, Quality )
			EndIf
			
		EndIf
	EndIf
	
EndProcedure

Procedure.i pds_create( )
	
	Define.PDS *this = AllocateMemory( SizeOf(PDS) )
	
	If *this
		InitializeStructure(*this, PDS)
		ProcedureReturn *this
	EndIf
	
EndProcedure

Procedure.i pds_destroy( *this.PDS )
	
	If *this
		If *this\contentDataAddress
			
			If IsImage(*this\bitmap)
				FreeImage(*this\bitmap)
			EndIf
			
			FreeMemory(*this\contentDataAddress)
			*this\contentDataAddress = #Null
			ClearStructure( *this, PDS )
			ProcedureReturn #True
			
		EndIf
	EndIf
	
EndProcedure

Testing images can be found at the NASA repository, a sample image is included here for studying purposes (Property of NASA).

Here is an example of how to load and convert an image:

	Define.PDS *file = pds_create()
	If *file
		pds_load( *file, "ab100402.img" )
		pds_plot( *file )
		
		If pds_save( *file, "output.bmp")
			RunProgram("output.bmp")
		EndIf
		
		pds_destroy(*file)
	EndIf

Have fun and stay safe!

Happy new year!

Posted by on December 31, 2014

Happy new year everyone!

Reviving an old Jacobs chuck – Electrolytic Rust Removal

Posted by on November 24, 2014

I’ve been using electrolytic rust removal for quite a while now, but this method never ceases to amaze me!

Here’s how I found the chuck, it came with a milling machine I acquired a few months ago, the previous owner had left some tooling in a rotten shed and this is the end result.

Jacobs chuck fossil

I was willing to bet this chuck was trash, and I was ready to throw it away. However it turned out to be an older type Jacobs, it was also a size I didn’t have… So I gave it a try, what can we lose right?

It was frozen shut. In theory the electrolysis should not affect the inside portion of the chuck, because it isn’t in the line of sight with the anode — but I had my hopes this could just work by freeing up the rust near the jaws and the collar.

Surprisingly after a few hours the chuck was in working order!

Sadly it's been abused (more than we thought) and it won't center within spec anymore, however it was quite an interesting experiment nonetheless!

Sadly it’s been abused (more than we thought) and it won’t center within spec anymore, however it was quite an interesting experiment nonetheless!

I did remove the arbor later on, not sure how they were using it like that, the inside of that arbor is hollow and threaded, but it has no surface to register with, aside from the flat face on the hexagonal portion — odd, maybe it was meant to be screwed onto a turret tool blank for a capstan lathe?

So there you have it, you CAN turn your archaeological finds into usable tooling, assuming no apes were previously involved.

Cheers,
Gus

Android – Debugging through WIFI

Posted by on February 9, 2014

Ever wondered if you could use ADB through WiFi to avoid using the USB connector while debugging on Android all the time?, turns out you can and it’s simpler than you thought!

Of course, the first step is to root your Android device, But don’t worry, it’s also very easy now adays. I recommend you try Framaroot — Yes, in theory there could be some risk involved, but given the current state of Framaroot I’ve never had a problem and it takes about 3 seconds to root your device, you don’t really need to reboot it either although it’s recommended.

Once you’ve got SU (SuperUser), you can proceed to install WiFi ADB — Avoid any other application, specially if it requires a host on your desktop / dev box since really, you’ve rooted your device, no need to go around obstacles that aren’t there anymore.

Now, simply start WiFI ADB and enable it, the final step is to get your ADB to listen to your device through TCP/ip, we simply open ADB via the console and perform the connect command followed by the ip and the port of your device.

For example in my case:

adb connect 192.168.2.100:5555

ADB should then report “connected”, otherwise make sure your device is listed to begin with using the “devices” command, it should pop the id of your device. Otherwise, make sure WiFI ADB is enabled.

That’s all!

DIY Electro-Etched Name plates and tags on steel

Posted by on September 20, 2013

The story goes…

I’ve been working on a metal project lately and I wanted to have a name / brand plate riveted to the project, after finding out how expensive a custom one can be I decided to try etching my own using very few tools and chemicals.

The process is similar to toner transfer PCB etching, however we’re going to be using sheet metal (mild steel) and we won’t be etching any traces, instead we’ll have our own logo, specs or any other information we want to add to our plate. We will also choose electrolysis over acids.

 

DSCN4084f

Personally I’m a huge fan of the “cast iron” look you can get with this technique, pretty neat huh?

 

Design it, print it, etch it.

Once you’ve got your black and white design ready, go ahead and print a mirror image to the desired dimensions.

  • Trim your sheet metal to size, mine was 1mm thick but you can use whatever you’ve got. Decide whether you want to leave an edge or not.

    DSCN4091f

    This design is actually an inside joke, I used a couple images from deviantart (hopefully they won’t mind?)

  • Clean the sheet metal, use a degreaser or common soap and a brush or pad to remove any oils and dirt it may have on it’s surface. Sorry, no pictures of this step.
  • You could choose whether to score the surface using 220 grit sand paper (or coarser) or just let the original surface be, it’s up to you. Sorry, no pictures of this step.
  • Proceed to toner-transfer as usual using your favorite method (a clothes iron works just fine).

    DSCN4094f

    Toner applied, ready for etching!

  • Prepare the electrolyte, washing soda and water is preferred but you can get away with salt and water; just do it outside and away from ferrous objects you may wish to remain intact (chlorine gas will rust them).
    • The amount depends on how much water you’re going to use, per liter I use 4 spoons of either chemical, in essence the more conductive your electrolyte is, the more current is allowed to flow and thus the faster the etching process becomes. I’m not entirely sure if the surface finish becomes rough with the increased current, you’ll have to experiment for yourself!
  • Use a battery charger or an ATX power supply, in fact any power supply capable of providing DC at several amps will do.

Your plate to be etched has to become the anode (positive) and the cathode (negative) terminal is connected to some scrap steel. This steel will actually become de-rusted, so maybe choose something you want to restore and kill two birds with one stone — Just try to match the surface area and keep the two parts evenly apart.

DSCN4098f

Etching… If you think the colour of the electrolyte is nasty, give it a few more minutes… The cathode is indeed too small, but it’s what I had. Plus I wanted to de-rust that part. Notice the steel wire used to contact the anode with the plate to be etched, try not to use copper.

 

Now it’s all about time…

I often leave the plate to etch for half an hour and I don’t care about the amps going through, I just remove it periodically and touch the surface to get a feeling of how much metal has been etched away.

When to stop is up to you, just keep in mind the resist (the toner) won’t protect the metal from the sides, so you’ll get what’s known as undercutting after a certain point.

There’s also the fact that if your current is high enough, the resist will begin to peel off. This could also happen if your part and electrolyte become overheated, so keep an eye on that matter.

DSCN4102f

Finally etched, it didn’t come out as I expected but it was a good way to learn, after all this was my first attempt!

 

Giving it a nicer look

Right now you can go ahead and rinse the plate you just etched. At this point there are several options on the finish…

DSCN4078f

This is the result after etching another design, no patina yet. So the contrast is low. There’s some undercut and portions where the resist peeled off; the current was too high.

You can leave it as-is for a natural finish, you can sand it with emery cloth or you can apply a layer of black oxide (hot or cold, up to you) and either leave it as-is or brush it to increase the contrast of the letters and to make it look aged.

Other options to create patinas include wood dyes, paint, etc. The work can be sealed with beeswax or a clear coat,

To black-oxide the plate without chemicals you’ll have to heat it red hot, then quench on linseed oil or motoroil. For a higher degree of control over the process you can choose to use a rag soaked in oil instead and apply that to the hot plate, just be careful and do this outside, the fumes are noxious and there’s always the risk of a fire, so keep sand or an ABC extinguisher nearby. This method requires multiple passes and takes a lot longer, but you can get any colour you want from gold to blue.

DSCN4084f

The end result, pretty good for the money! — All that remains now is a simple cut and a couple holes to be drilled, a countersink wouldn’t hurt either.

 

If all went well, you’ll have a nice looking plate for your current project, or just some pretty cool piece of art. Don’t get stressed out if it didn’t come out exactly how you wanted it to, it’s still usable and unique.

Get creative and have fun!

Cheers.