Unity - Panorama Viewer

From NoskeWiki
Jump to navigation Jump to search

About

NOTE: This page is a daughter page of: Unity and Cinema 4D


Photosphere process.

On this page is a tutorial to setup interactive panorama (aka. photosphere) viewer with sound, suitable for Google Cardboard, using the Unity game engine. Unity was chosen because it's an incredible popular authoring tool for mobile, very quick for prototype and works on multiple platforms. The code below (less than 200 lines of C#) has been tested on: iPhone, Android and the web.


HowTo: Make an Easy Photosphere Viewer in Unity

Google photospheres are saved in a special format (aspect ratio 2:1 using to Equirectangular_projection where the top and bottom appear stretched) designed to wrap around a sphere. Unity already has a sphere object, but since we want to see the inside of a sphere, we need a sphere with inverted normals so that we can see the photosphere from the middle.

  • Create a new project in Unity.
  • Add InvertedSphere script: Add this "InvertedSphere.cs" C# script file under YourProject/Assets/Editor/ :

YourProject/Assets/Editor/InvertedSphere.cs: 

using UnityEngine;
using UnityEditor;
 
public class InvertedSphere : EditorWindow {
  private string st = "1.0";
 
  [MenuItem("GameObject/Create Other/Inverted Sphere...")]
  public static void ShowWindow() {
    EditorWindow.GetWindow(typeof(InvertedSphere));
  }
 
  public void OnGUI() {
    GUILayout.Label("Enter sphere size:");
    st = GUILayout.TextField (st);
 
    float f;
    if (!float.TryParse (st, out f))
      f = 1.0f;
    if (GUILayout.Button("Create Inverted Sphere")) {
      CreateInvertedSphere(f);
    }
  }
 
  private void CreateInvertedSphere(float size) {
    GameObject go = GameObject.CreatePrimitive(PrimitiveType.Sphere);
    MeshFilter mf = go.GetComponent<MeshFilter>();
    Mesh mesh = mf.sharedMesh;
 
    GameObject goNew = new GameObject();  
    goNew.name = "Inverted Sphere";
    MeshFilter mfNew = goNew.AddComponent<MeshFilter>();
    mfNew.sharedMesh = new Mesh();
 
 
    //Scale the vertices;
    Vector3[] vertices = mesh.vertices;
    for (int i = 0; i < vertices.Length; i++)
      vertices[i] = vertices[i] * size;
    mfNew.sharedMesh.vertices = vertices;
 
    // Reverse the triangles
    int[] triangles = mesh.triangles;
    for (int i = 0; i < triangles.Length; i += 3) {
      int t = triangles[i];
      triangles[i] = triangles[i+2];
      triangles[i+2] = t;
    }
    mfNew.sharedMesh.triangles = triangles;
 
    // Reverse the normals;
    Vector3[] normals = mesh.normals;
    for (int i = 0; i < normals.Length; i++)
      normals[i] = -normals[i];
    mfNew.sharedMesh.normals = normals;
 
 
    mfNew.sharedMesh.uv = mesh.uv;
    mfNew.sharedMesh.uv2 = mesh.uv2;
    mfNew.sharedMesh.RecalculateBounds();
 
    // Add the same material that the original sphere used
    MeshRenderer mr = goNew.AddComponent<MeshRenderer>();
    mr.sharedMaterial = go.renderer.sharedMaterial;
 
    DestroyImmediate(go);
  }
}
  • Add an inverted sphere: Immediately after adding this script, click "Menu > GameObject > Insert Other > Inverted Sphere..." then give a radius (say 20), and center it at origin (position: 0,0,0).
  • Download a photosphere jpg: Go to your Google Photos account, find a photo-sphere and download it, or download the sample attached to this page (here). The name should look like: "PANO_20140904_123716.jpg".
Photosphere / pano example
  • Make this a texture in unity: Create a YourProject/Assets/Textures/ folder and add this file. In Unity's project window, find the same file, selected it, and then in it's Inspector window change the max size to "4096" then Apply (it won't go bigger sadly, so you'll still lose some res, but not as much).
  • Make a material: In the same folder click "Create > Material". Give it a matching name, then drag the JPG pano texture into it's "BASE" (from the Project window in the Inspector Window).
  • Add the material to the inverted sphere: Select the inverted sphere in the Hierarchy window then (in the Inspector window) drag the material under "Inspector window > Mesh Renderer > Materials > Element 0", drag it again as the 2D texture and change the Shader to "Unlit/Texture".
Photosphere
  • Setup the scene: Change the position "Main Camera" to origin, and preview this in the Game panel by clicking Play. You should be able to change the camera's "y" rotation and you'll turn within the photosphere. You might also want to add a Point Light to, but if the photosphere rendering is "Unlit" then this won't make a difference.



HowTo: Make Your PhotoSphere Rotate

A photosphere may as well be a static image until you enable rotation. When you hold down the mouse we want to be able to look left/right and up/down. One approach is to rotate the sphere while the camera stay still, but the more versatile and "correct" approach is to have the camera rotate and leave the sphere static. Following the previous instructions, here is what to do:

  • Add the following C# script file under YourProject/Assets/Scripts :

YourProject/Assets/Scripts/CameraMovement.cs: 

using UnityEngine;
using System.Collections;

public class CameraMovement : MonoBehaviour {
  public float rotateSpeed = 100.0f;
	
  void Update () {
    if (Input.GetMouseButton(0)) {  // If left mouse button down:
      float rotateAboutX = Input.GetAxis("Mouse Y") * Time.deltaTime * rotateSpeed;    // Mouse movement up/down.
      float rotateAboutY = -Input.GetAxis("Mouse X")  * Time.deltaTime * rotateSpeed;  // Mouse movement right/left.
      gameObject.transform.Rotate(rotateAboutX, rotateAboutY, 0.0f);   // Rotate the (camera) object around X and Y.
	  
	 // Lock z rotation to 0 so camera doesn't 'roll' sideways.
	 // We only want the camera's yaw (y) and pitch (x) to change.
      var newRotation = gameObject.transform.rotation.eulerAngles;
	  newRotation.z = 0;
	  gameObject.transform.rotation = Quaternion.Euler(newRotation);
    }
  }
}
  • Now drag this script onto your MainCamera.
  • Hit play, and experiment... you may want to change the rotateSpeed value. Notice you don't need to edit code, you can change it in the Inspector window, since this variable is marked public.
  • If instead you want to rotate with the accelerator of a phone, change the "Input.GetAxis("Mouse X")" to Input.acceleration.x as per this example.


HowTo: Make Your Camera Rotate with the Phone (Using the Gyroscope)

If you want to be able to move your phone in all angles and see that direction you only need to call "Input.gyro.enabled" at the start then use the "Input.gyro.attitude" Quaternion.

Unfortunately, this figuring out how to transfer this Quaternion is really hard - lots of the pieces of code on the web I found didn't work - and it took me forever to work out. The code below works for an Android (iPhone uses left hand instead of right hand) built to LandscapeLeft orientation (the phone on its side). If using iPhone or a different orientation you'll have to play with the values... and unless you're a Quaternion expect it will take up much of your day! To test this example, remove the "CameraMovement" script from "Main Camera", and add this one instead. Then you'll have to test it on your Android by following these instructions. 

using UnityEngine;
using System.Collections;

public class CameraRotateUsingPhoneGyro : MonoBehaviour {
  void Awake () {
    // Check gyro works on this device:
    if (!SystemInfo.supportsGyroscope) {
      Debug.Log ("WARNING: Gyro not supported");
      if (Application.platform != RuntimePlatform.IPhonePlayer ||
        Application.platform != RuntimePlatform.Android) {
        Debug.Log ("WARNING: Gyro only works on phone");
      }
    }

    Input.gyro.enabled = true;    // Enable gyro.
  }
  
  void Update () {
    // The following code works on Android with the orientation set to "LandscapeLeft"
    // (under "Edit > Player Settings > Player..."). The Android gyro is a little funny and
    // here the z and w values of the quaternion are inverted, and the whole thing effectively
    // rotated up (in pitch) by 90 degrees. For iPhone or for different orientations you'll
    // have to carefully inspect the Euler angles, think about yaw pitch roll, change values
    // and search the internet for a correct solution. Lots of the internet solutions failed
    // for me and Quaterions are hard!
    Quaternion transQuat = new Quaternion(Input.gyro.attitude.x, Input.gyro.attitude.y,
                                         -Input.gyro.attitude.z, -Input.gyro.attitude.w);
    gameObject.transform.rotation = Quaternion.Euler (90, 0, 0) * transQuat;
    Debug.Log ("transQuat = " + transQuat.eulerAngles.ToString());
  }
}


HowTo: Add 3D Sound

  • You might want to put different sounds at different locations in your photosphere. Using the provided photosphere, notice that you initially face the ocean, but behind you is island forest. These instructions will help you put an ocean noise in front and rainforest noise behind.
  • Download the two sound files attached at the bottom of this document. Create a folder YourProject/Assets/Sounds and drop all your sounds in here.
  • Drag the "OceanSound" from the Projects window directly into the Scene and set it to position (10,0,10) notice it is automatically a 3D sound, and we want it on the ocean side.
  • Drag the "RainforestSound" in also, and set to position (-10,0,10).
  • Click play and then notice the sounds seem to change as you spin around.
  • To further exaggerate there effects, consider writing a script to reduce the volume of any sound the camera is facing directly away from (measure the angle difference).
Adding 3D sound to scene


HowTo: Split the Screen for Side-by-Side Stereo Pair

  • To work with VR Googles, you might want to split the screen to represent the left and right eye. Usually we'd want the left and right eyes to see the scene from slightly different positions and simulate 3D vision (our eyes are an average 65mm apart - enough to add useful depth perception), but since the photo-sphere is just a sphere with no depth information we'll make the left and right side identical. Splitting the screen in Unity is incredibly easy. These instructions continue from the previous tutorial, where we have a "Main Camera" set at origin (0,0,0) and default rotation (0,0,0).
  • Click "Menu > GameObject > Create Other > Camera". Rename this camera to "Left Eye" and in the Inspector window, right click the "Audio listener" and select "Remove Component". We only want one listener, and we'll leave that on the Main Camera. Set the Left Eye to origin, and default rotation (matching our Main Camera), then make it a child of "Main Camera". In the Inspector Window, change the Viewport Rect of Left Eye to have W of 0.5, with X still 0.
  • In the Hierarchy window, right click the Left Eye and click Duplicate. Call the second one "Right Eye", and give it a W of 0.5 and X of 0.5.
  • Click Play and you should see side-by-side. Notice that if you disable the Left Eye and Right Eye you can still the normal "Main Camera" operating underneath as normal. If you want, you could create a script to toggle these two eye Cameras on and off, and thus toggle the split screen.

NOTES:

  • Normally you would separate the left and right eye apart (say 3 centimeters offset left and right). This works inside 3D scenes, but since we haven't applied real depth to our photosphere (it's just a sphere), we put the left and right eye at the same spot.
  • To see a wider area, simply play with the "field of view" of both Eye cameras. This could be set to a script to effectively "zoom in" and "zoom out", although I wouldn't suggest zooming out much further than 100 degrees.


Split screen setup


Here's a script to toggle split screen: 

using UnityEngine;
using System.Collections;

public class Settings : MonoBehaviour {
  private Camera leftEye;
  private Camera rightEye;
  private bool splitScreenOn = true;

  void Awake () {
    leftEye = GameObject.Find ("Left Eye").camera;
    rightEye = GameObject.Find ("Right Eye").camera;
  }	

  void Update () {
    if (Input.GetKeyUp (KeyCode.S)) {
      splitScreenOn = !splitScreenOn;
      leftEye.enabled = splitScreenOn;
      rightEye.enabled = splitScreenOn;	
    }
  }
}


Code license
For all of the code on my site... if there are specific instruction or licence comments please leave them in. If you copy my code with minimum modifications to another webpage, or into any code other people will see I would love an acknowledgment to my site.... otherwise, the license for this code is more-or-less WTFPL (do what you want)! If only copying <20 lines, then don't bother. That said - if you'd like to add a web-link to my site www.andrewnoske.com or (better yet) the specific page with code, that's a really sweet gestures! Links to the page may be useful to yourself or your users and helps increase traffic to my site. Hope my code is useful! :)



See Also

Retrieved from "http://andrewnoske.com/w/index.php?title=Unity_-_Panorama_Viewer&oldid=1292"