Android and Augmented Reality I

Hi everybody,

i think in the last decade, one of the most exciting trend in Mobile development world was augmented reality. Today i will try to explain how augmented reality works and how to write your very own augmented reality engine. This article will be in three parts:

1. The first chapter will cover creating surfaceView, placing the camera preview on the surface and adding one more transparant view on the surface.
2. Explanation about getting latitude, longitude, azimuth and pitch and some more information about sensors.
3. This chapter covers openGL, calculating the x, y and z indexes of a shape and translate it's 3d position on the camera view.

We will use the following permissions;

android.permission.CAMERA
android.permission.ACCESS_GPS
android.permission.ACCESS_ASSISTED_GPS
android.permission.ACCESS_LOCATION
android.permission.ACCESS_COARSE_LOCATION
android.permission.ACCESS_FINE_LOCATION

But just the camera permission is needed for the first chapter. So there are three important components in our activity. First, surfaceView, second surfaceViewListener and third the camera. We don't have to specify a callback for camera since we never will take a picture. We are going to create a surface, listen it and once the surface is created set the surface as preview display of the camera.

So this is how our activiy will look like;

public class AugmentedReality extends Activity{
private View mainView;
private View appView;
private MyCameraView cv;


@Override
public void onCreate(Bundle savedInstanceState) {
//as eveybody says, this will run first :)
super.onCreate(savedInstanceState);
//the main container for the application. We will overlay two views here
//this element should be defined in the XML file
mainView = findViewById(R.id.main);
//create another transparant view
appView = new View(this.getApplicationContext());
//no background color and resource so the holder of the view will be totally invisible
appView.setBackgroundcolor(0);
appView.setBackgroundResource(0);
//create a local variable for the camerra view
//we will define this class in the activity
cv = new MyCameraView(this.getApplicationContext());
//add it to the main view
mainView.addView(cv);
//now we add the second view so it will take the highest z index automatically
mainView.addView(appView);
}
//now we are outside of onCreate.
//lets define the CameraView
public class MyCameraView extends SurfaceView
{
//lets build the constructor
public CustomCameraView(Context ctx)
{
super(ctx);
//we get the holder
previewHolder = this.getHolder();
//set the type
previewHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
//and add the listener which is defined below
previewHolder.addCallback(surfaceHolderListener);
}
}
//now we are going to define the listener
//and trigger the camera preview once the surface is visually created
SurfaceHolder.Callback surfaceHolderListener = new SurfaceHolder.Callback() {
private Camera camera;
public void surfaceCreated(SurfaceHolder holder) {
//so the surface is created, we can open the camera
camera = Camera.open();
Parameters params = camera.getParameters();
//we set the format of the preview
//so it will be encoded
//if you get colored horizontal lines instead of preview
//this is the reason
params.setPreviewFormat(PixelFormat.JPEG);
//we don't have to define the format of the taken picture
//since we are not going to take a picture
//params.setPictureFormat(PixelFormat.JPEG);

//finally setthe new parameters
camera.setParameters(params);
try {
//set the surface where the preview should be displayed
camera.setPreviewDisplay(holder);
camera.startPreview();
} catch (IOException e1) {
e1.printStackTrace();
}
}
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height)
{
//you can add your own function here
//this event is fired when any value of the surface is changed
//for example changing to landscape mode automatically
}

public void surfaceDestroyed(SurfaceHolder arg0)
{
//ok when the application is shutdown, we release the camera
//so other applications can use it too
camera.stopPreview();
camera.release();
}
};
}
}

This code will give us the preview of the camera on a surface. In the next chapter we will dive into sensors and retrieve the needed data like latitude, longitude, azimuth and pitch. Thanks for reading.