Quirks on Silverlight 5 with GPU acceleration
One of the much anticipated features of Silverlight 5 was 3D hardware acceleration. In this age of social networking in the browser, it’s great to finally be able to develop 3D apps/games for this crowd.
Unfortunately, we are disappointed with some of the solutions the Silverlight team came up with in this integration. These will affect you when developing for Silverlight, and some will also affect the experience by the end user…
Dynamically generate a test page
When running your Sliverlight application in the browser, you need to have an HTML page that embeds the control and loads your application. If the application has 3D graphics, you need to add the following parameter to the HTML:
<param name="EnableGPUAcceleration" value="true" />
Fortunately, Visual Studio has a “dynamically generate a test page” option in the project properties, saving you some development time. Unfortunately, they didn’t add the required parameter in the page generator, making the generated page pointless.
We have created our own page generator that fixes this issue. When using the YVision project templates for Silverlight 5, you find it called as a post-build event. To use the generated page when debugging, open the project properties, click on the “Debug” tab and then check the “Specific Page” radio button. Click on the “…” button and select the .html page that is now part of the project.
User permisson required to use GPU acceleration
We all know that the browser has the great responsibility of only executing secure code. The 3D graphics depend on an essential third-party component, the graphics board driver. These drivers may have bugs that can be exploited to execute malicious code.
It’s fair that Silverlight should ask for permission to use the GPU, just like it does for the microphone and the webcam. Unfortunately, the Silverlight team chose a solution that requires at least 8 mouse clicks and no explanation at all.
When entering your web site the user will have to:
- Right click on the Silverlight plugin and a context menu will appear.
- Click on the “Silverlight” menu item.
- Open to the “Permissions” tab.
- Look for the host address of the page you’re visiting.
- Select the “3D Graphics: use blocked display drivers” option , that should have “Denied” right in front of it.
- Click on the button “Allow” located after the list.
- Click on “OK” top close the dialog and then (at least with YVision/Axiom) you’ll have to reload the page.
I really hope there will be a Silverlight 5.1 that shows a message and requires a single click. Just like it does for the webcam and the microphone. It’s that simple…
No support for GPU acceleration outside Windows
Silverlight runs on the major browser both on Windows and Mac OS X but, 3D accelerated graphics are only supported in Windows.
I can’t find any official information about this. If you google for this information you’ll only find people asking this same question. The closest I can find is “At Mix11, speaker said that 3D support was being considered [for Mac] and they knew it was important to many. So, not-committed.”
We would love to have an official stand on these three issues!…
YVision 4.0 Beta 3 Released!
It’s been a long time since out last release. We have been busy replacing some of the engines to make sure YVision can execute on as many as possible platforms. This work paid off with better support for Silverlight 5 in this release and mobile devices support in the near future.
Check out the features in this latest release:
Axiom on Windows Phone 7
Axiom is a port to C# of the popular Ogre render engine. We are helping this open-source project to support environments that don’t allow the “unsafe” keyword (the use of pointers). The video shows a build of the latest source running on a HTC HD7 device.
We will start now working on adding support to YVision for Windows Phone 7 and Silverlight 5, all with the same rendering engine we are using for Windows and Mac OS X.
YVision is going cross-platform
The YVision platform includes several features that are huge undertakes by themselves. These are features like computer vision, rendering, physics, etc. We want to deliver an innovative development platform but we don’t waste time “reinventing the wheel”. Specially when there are very dedicated people delivering amazing projects as open-source. YVision is not open-source but it integrates several open-source projects.
YVision is developed on .NET and, when we started developing it, it was impossible to find .NET specific projects in these areas. There were only native (non .NET) projects and very few had sister projects for .NET wrappers, enabling their use in .NET projects.
For many years we integrated OpenCV, Ogre and ODE as our primary engines, with their .NET wrappers MOGRE and OdeNet. We didn’t find any OpenCV wrapper with a suitable license so we developed our own, that is also included in the current release of YVision. For video input, we use the DirectShow.NET wrapper.
When we started developing YVision 4.0 we added a new requisite. It has to execute cross-platform and have as little platform-specific code as possible.
.NET is compatible with Windows, Silverlight and Windows Phone 7 but, with Mono (one more awesome open-source project), it’s possible to have .NET applications executing in Mac OS X and Linux, and with Xamarin‘s products, in iOS and Android.
The YVision core is compatible with .NET and Mono, making it compatible with all the mentioned platforms. The challenge was to have third-party engines support all these platforms, as each one of these have its own restrictions.
Mono supports P/Invoke (used by many wrappers) but doesn’t support mixed-mode assemblies (used by some wrappers, like Mogre). Silverlight and the mobile platforms support none of these (Silverlight 5 supports P/Invoke but “Out-Of-Browser”).
The solution to have as little platform specific code as possible is to have fully managed engines (.NET specific code). We looked around and found some good candidates that have already been integrated into YVision 4.0.
Axiom is a port to C# of the Ogre rendering engine. It’s a bit behind in terms of Ogre features but it’s fully managed code, Mono compatible and supports DirectX, OpenGL and XNA rendering systems. It makes extensive use of “unsafe” (a common C# performance optimization) that unfortunately is not supported in Silverlight and Windows Phone 7. We’ve been actively contributing code to fix this limitation. We are experiencing very reasonable frame rates on these platforms, and we hope to have demos very soon.
bullet-xna is a port to C# of the Bullet physics engine. It’s fully managed code but it depends on the XNA math library that is not available in all the platforms. We’ve also been actively contributing code to replace this library by the Mono.GameMath, that is compatible with all platforms.
We’ve also been contributing to SLARToolkit, a fully managed code augmented reality library. For example, we have contributed with an adaptive threshold that makes tracking much more robust to lighting changes. We are waiting for the binary release and approval of some more contributions (Right, René? 
).
We are looking to other projects like AForge, to replace OpenCV, and GStreamer, to replace DirectShow.NET.
Currently YVision can be developed and executed on Windows and Mac OS X. Next, we will support Windows Phone 7 and we will upgrade the support of Silverlight 5 to the RTM release (currently depending on the Axiom release). In the near future we will start working on support for iOS and Android.
YVision for Mac now with support for Lion and MonoDevelop 2.8
We’ve just released version 4.0.2.1 of YVision for Mac OS X.
This version includes the following new features and improvements:
- Support for Mac OS X Lion.
- Upgraded MonoDevelop integration for MonoDevelop 2.8
- Mac specific dialogs for render settings and camera selection.
- Several performance improvements that doubled the frame rate of the samples.
Get this update from the download page and follow the simple instructions on how to install it.
Notice that if you have MonoDevelop 2.6 installed, you’ll have to upgrade to 2.8
Mac OS X is a trademark of Apple Inc.
YVision development now also supported on Mac OS X
We are committed to a monthly release with bug fixes, new features and support for more platforms. This month we are happy to announce support for development on Mac OS X.
YVision for Mac OS X is distributed as an add-in for MonoDevelop 2.6, a free open-source integrated development environment (IDE). Mac developers can now take advantage of a modern component-based architecture for the development of robust Natural User Interfaces (NUI) on a powerful and easy to use IDE, with code templates, color highlighting, code completion, integrated debugger and much more.
Just like in other platforms, YVision for Mac OS X features:
- real-time 3D rendering
- 3D physics simulation
- video input handling
- input devices handling
- computer vision
- multi-tasking
Get it from the download page and follow the simple instructions on how to install.
Mac OS is a trademark of Apple Inc.
YVision 4.0 Beta 2 is available for download!
We are proud to announce the release of one more version of the YVision SDK. This is part of an aggressive schedule of frequent upgrades with new functionalities and new target platforms:
- Silverlight 5 Beta integration: it is now possible to create Silverlight YVision projects with the help of Visual Studio project templates.
- SLARToolkit integration: allow the creation of marker-based Augmented Reality applications on the web.
- New tutorials:
- Tutorial 2 – Behaviors 101: explains the ins and outs of the YVision Behavior concept. Don’t forget to check the project template for this tutorial.
- New Visual Studio item templates:
- Behaviors
- EmptyBehavior
- ActiveIterationCondition
- BehaviorTree
- CooperationDecorator
- InvocationDecorator
- CompositeBehavior
- CompositeCooperate
- CompositeInvoke
- ConditionBehavior
- Templates:
- WorldObject Template
- CompositeTemplate
- New Visual Studio project templates:
- BasicApplication
- BasicApplicationSilverlight
- BoxFall
- BoxFallSilverlight
- MarkerSilverlight: sample of Augmented Reality using SLARToolkit
- Improved camera selection dialog for windows apps.
- Bug fixing.
- Class documentation.
You can find all this on our download page!
Don’t forget to check the support page to give us feedback or solve any problem you may find.
Santader Robots featured in Wired UK
The Santander Robots, developed by YDreams and powered by YVision, were covered in a Wired article.
We are very happy with this case, because it represents a really complex use of our behavior trees. I would probably be impossible to incorporate and process so much complex sensor data without YVision’s “holistic” approach.
About the robots:
“These intelligent robots are here to help. Called Sigas (Santander Interactive Guest Assistants), the five 61cm-tall robots guide guests around the visitors’ centre of Santander bank’s Group City HQ, near Madrid. Using 16 sonar sensors for object detection and RF to take bearings from the centre’s 12 RF tags, each robot is autonomous, sharing information with the others to assign tasks efficiently. In their down time, the Sigas play music, interact with people or chat among themselves.”
YVision 4 Beta 1 is out now!
We are proud to announce the release of the first public beta of YVision. It’s free, so please join us in the NUI revolution.
To download please register at our page:
www.yvision.com
This beta release is for PC only, but we are working hard in testing the Mac and Silverlight versions for the next release. We intend to “release often”, so please keep posted!
Cheers!
The YVision team
What is YVision?
YVision is a development framework that enables rapid prototyping and development of applications based on Natural User Interfaces (NUI). It integrates computer vision, real-time rendering, physics simulation, augmented reality, artificial intelligence, multitasking and much more.
Augmented Reality
YVision has already been used in hundreds of projects and many of these were Augmented Reality applications. Check out this show reel with some of the applications developed for YDreams customers or made by the R&D lab on the quest to create new forms of interaction.
Overview of YDreams work & research in Augmented Reality from YDreams on Vimeo.