Facultatea de Automatica si Calculatoare Page 1
Universitatea "Politehnica" din Timişoara Facultatea de Automatică şi Calculatoare Departamentul Calculatoare 2, Vasile Pârvan Bv., 300223 – Timişoara, Romania Tel: +40 256 403261, Fax: +40 256 403214 Web: http://www.cs.upt.ro
Diploma Thesis
Providing interactive, cost-efficient and engaging educational science experiments by using IQubes, off-the-shelf devices and
the cloud
Coordinator: Conf.dr.eng. Marius George MARCU
Cristian-Viktor ARDELEAN
Lucian Alexandru BARA Remus Gabriel BARBATEI
June 2012 Timisoara
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 2
Confucius
I hear I forget. I see I remember. I do I understand.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 3
Table of Contents Motivation..................................................................................................................................................... 6
Executive summary ....................................................................................................................................... 7
Introduction .................................................................................................................................................. 8
Theoretical aspects ....................................................................................................................................... 9
Architecture ................................................................................................................................................ 12
Implementation .......................................................................................................................................... 13
ASP.NET MVC3 Web App ............................................................................................................................ 15
1.Introduction ......................................................................................................................................... 15
Controllers............................................................................................................................................... 16
Views ....................................................................................................................................................... 18
Access Control Service (Windows Azure AppFabric’s ACS) ..................................................................... 20
Windows Live Login Client ...................................................................................................................... 23
User Interface ......................................................................................................................................... 25
Educator Interface in Silverlight for Embedded .......................................................................................... 29
Silverlight for Windows Embedded ........................................................................................................ 29
Windows Embedded Silverlight Tool with Microsoft Expression Blend ................................................. 30
Abstract ....................................................................................................................................................... 37
Introduction ................................................................................................................................................ 38
Theoretical aspects ..................................................................................................................................... 40
Basic workflow aspects ........................................................................................................................... 40
Handling Exceptions ................................................................................................................................ 45
Custom Activities .................................................................................................................................... 45
Persistence and Storage .......................................................................................................................... 46
Workflow Tracking and Tracing .............................................................................................................. 47
Rehosted designer .................................................................................................................................. 48
Windows Phone 7 software development ................................................................................................. 49
Sensors .................................................................................................................................................... 51
IronRuby .................................................................................................................................................. 51
Implementing the workflows ...................................................................................................................... 53
Activity Library ........................................................................................................................................ 54
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 4
Rehosted designer .................................................................................................................................. 56
Implementing tracking ............................................................................................................................ 61
Implementing the Windows Phone 7 Client ............................................................................................... 64
Database ................................................................................................................................................. 64
User interface.......................................................................................................................................... 65
Sensors .................................................................................................................................................... 66
Cloud ....................................................................................................................................................... 66
Conclusions ................................................................................................................................................. 68
Abstract ....................................................................................................................................................... 69
Introduction ................................................................................................................................................ 70
Device description ....................................................................................................................................... 72
Mechanical aspects ................................................................................................................................. 72
Electronic aspects ................................................................................................................................... 72
Architecture ................................................................................................................................................ 73
Embedded Device Hardware Modules ....................................................................................................... 75
Internal .................................................................................................................................................... 75
Input ........................................................................................................................................................ 77
Output ..................................................................................................................................................... 78
Control .................................................................................................................................................... 80
Embedded Device Application .................................................................................................................... 81
SD Card and FAT system ......................................................................................................................... 81
SPI Mode ............................................................................................................................................. 81
FAT16 file allocation table - 16bit ....................................................................................................... 87
Short Overview ................................................................................................................................... 87
SDRAM Memory...................................................................................................................................... 88
Embedded application for the Hall-effect sensors ................................................................................. 90
Digital to analog converter and wave file format ................................................................................... 91
LCD display embedded application ......................................................................................................... 92
Communication with the coordinator .................................................................................................... 92
Protocol and Data flow ............................................................................................................................... 94
IQube to Coordinator .......................................................................................................................... 94
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 5
Coordinator to IQube .......................................................................................................................... 95
Coordinator Application .............................................................................................................................. 97
Similar products and costs .......................................................................................................................... 98
Similar products ...................................................................................................................................... 98
Costs of our solution ............................................................................................................................... 99
Iqube games .............................................................................................................................................. 100
Tools and software used ........................................................................................................................... 102
Hardware .............................................................................................................................................. 102
AVR Dragon programmer .................................................................................................................. 102
Software ................................................................................................................................................ 102
DipTrace ............................................................................................................................................ 102
Realterm ............................................................................................................................................ 102
Audacity ............................................................................................................................................ 102
AVR Studio® 5.................................................................................................................................... 103
CATIA V5R19 ..................................................................................................................................... 103
Adobe Photoshop CS5 ....................................................................................................................... 103
Implementation, testing and future work ................................................................................................ 104
Future improvements ........................................................................................................................... 104
Wireless charging .............................................................................................................................. 104
Consumer product ............................................................................................................................ 104
Conclusions ............................................................................................................................................... 106
Bibliography .............................................................................................................................................. 107
Windows Communication Foundation Service ................................................................................. 107
Windows Workflow Foundation interoperability with embedded devices...................................... 107
IQube: a cube for learning – embedded educational device ............................................................ 108
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 6
Motivation
One of the major problems we identified is that children do not have the same opportunities
in terms of laboratory equipment. The United Nations Millennium Development Goal of
Universal Education mentions: ―Inequality thwarts progress towards universal education‖.
As stated in [HSLWE01] the American Indian Charter School, an Oakland, California ―has
no lab equipment; science is taught mostly through textbooks‖.
According to [8] the schools in Rawalpindi, India lack on basic school equipment and ―for
the last 20 years no special grants have been made to provide modern equipment for the labs,‖
stated a teacher, requesting anonymity [HSLWE01] [VNLW01][VTYW02][RMPB01].
He also was concerned that ―In the absence of practical studies, students face difficulty in
passing the examination and mainly depend on the efforts of their teachers‖ [HSLWE01]
Thomas Kalil (Deputy Director for Policy for the White House Office of Science and
Technology Policy) also emphasizes the importance of practical experiments and study in the
curricula (as stating in [WGCSP02]): ―We also have an exciting opportunity to give every child
the ability to become a Maker and to have the experience of designing and building something
that they find personally meaningful to them [...]‖. He also talked about ―curiosity- driven‖
research, considering it a big pillar in the development of human knowledge: ―Support for
curiosity-driven research is critical, both because expanding the frontiers of human knowledge is
an end in itself, and because it leads to benefits that we could never have predicted.‖
Jan Gahala, M.A., technical specialist in NCREL's Communications department raises a
series of obstacles teachers face regarding the implementation of technology, like computers in
schools: ―Hardware and software often pose problems for teachers in the classroom, and just-in-
time technical support may be unavailable. Teachers may lack the time and the motivation to
learn technology skills. Professional development activities may not provide ongoing, hands-on
training for teachers or practical strategies for implementing technology into lesson plans. Initial
technology funding may not be sustained and thus not capable of providing upgrades,
maintenance, and ongoing professional development.‖[JGPT03]
Also the costs for equipping a laboratory raise up to 2000 - 3000 (USD) and there are also
additional marginal cost of 200 - 300 (USD) according to a teacher from a school in Rawalpindi,
India[HSLWE01]
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 7
Executive summary
With our innovative LiveX Learning Platform, we strive reduce the inequality in universal
education and help the elementary schools from all countries, but greatly benefiting the ones
from poor countries, to deliver experiments in many different scientific disciplines at a much
lower and affordable budget.
Many children will have the chance to interact in a more engaging and creative way with
science and understand it much better by experimenting with what they learn, rather than just
reading textbooks.
Teachers will have the possibility to create custom experiments and share them with other
schools by publishing them on our experiment marketplace, creating a community around our
solution.
We are fully aware that no simulation or virtual experiment can truly offer the same
experience as a real-life carried out experiment with real objects, but our platform is somewhere
between the pure virtual world and the practical world.
Moreover, our platform provides education to be accessible from any location, anywhere in
any time with countless possibilities of improvement.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 8
Introduction
Our proposed solution to this problem consists of a Live Experiment Software Platform
(LiveX) running in the cloud, hosting experiments in areas such as chemistry, mathematics and
physics (optics, electronics, mechanics, etc.). Generic terminal devices working with our
software platform can be connected in various creative ways in order to simulate an experiment.
Considering the lack of IT staff for maintenance and programming, having a cloud solution
makes sense as it shifts responsibility to a 3rd party allowing the school to focus on other tasks.
Cloud also means you have the same availability and experience literally anywhere in the world,
which comes to answer the problem of a fragmented and unequal school system. For the cloud
provider, Microsoft Windows Azure is a good choice, since we noticed that the probable usage
patterns fit well with the Azure workload patterns. It also already adopted in some parts of the
world [WIAM03], thus our platform comes as a natural expansion.
Our solution comes to aid pupils and teachers around the world. It is of great benefit not
only to underdeveloped countries but also to developed countries. Although, in some cases
traditional learning methods are very effective, in some areas, especially experimental ones, they
lack the concept of new and up-to-date. Off-the-shelf also, means that the hardware prices can
range from very cheap to relatively expensive depending on school funding, received donations
and aids. In addition, our platform does not need homogeneous hardware, which is a case with
donated equipment. Children who already live in a modern environment will have the chance to
leave traditional learning methods and experience something new, attractive to them and more
closely related to the other gadgets they use in their after-school time, allowing them to use
gadgets they may own for experiments.
Our vision is to provide the possibility of performing scientific experiments virtually
anytime and everywhere. For this, you only need Internet access and a few off the shelf end
devices such as laptops/tablets, phones running on Windows Phone 7.1.1, Kinect or universal,
custom and low price LCD cubes made by us (IQubes).
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 9
Windows Communication Foundation Service
Theoretical aspects
Windows Communication Foundation (WCF) represents a framework for creating service-
oriented applications and send asynchronous messages from one service endpoint to another.
Messages can be complex binary streams or strings sent in XML format.
In the LiveX software platform all the data and the logic of the experiments accessed
through end devices are stored in the cloud. This aspect places a WCF Service at the heart of our
platform. End devices such as mobile phones, tablets, etc. are accessing methods exposed by the
WCF service.
Windows Communication Foundation offers the following set of features[EWSL09]:
Service Orientation
One consequence of using WS standards is that WCF enables you to create service
orientedapplications. Service-oriented architecture (SOA) is the reliance on Web services to send
and receive data. The services have the general advantage of being loosely-coupled instead of
hard-coded from one application to another. A loosely-coupled relationship implies that any
client created on any platform can connect to any service as long as the essential contracts are
met.
Interoperability
WCF implements modern industry standards for Web service interoperability.
Multiple Message Patterns
Messages are exchanged in one of several patterns. The most common pattern is the
request/reply pattern, where one endpoint requests data from a second endpoint. The second
endpoint replies. There are other patterns such as a one-way message in which a single endpoint
sends a message without any expectation of a reply. A more complex pattern is the duplex
exchange pattern where two endpoints establish a connection and send data back and forth,
similar to an instant messaging program.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 10
Service Metadata
WCF supports publishing service metadata using formats specified in industry standards
such as WSDL, XML Schema and WS-Policy. This metadata can be used to automatically
generate and configure clients for accessing WCF services. Metadata can be published over
HTTP and HTTPS or using the Web Service Metadata Exchange standard.
Data Contracts
Because WCF is built using the .NET Framework, it also includes code-friendly methods
of supplying the contracts you want to enforce. One of the universal types of contracts is the data
contract. In essence, as you code your service using Visual C# or Visual Basic, the easiest way to
handle data is by creating classes that represent a data entity with properties that belong to the
data entity. WCF includes a comprehensive system for working with data in this easy manner.
Once you have created the classes that represent data, your service automatically generates the
metadata that allows clients to comply with the data types you have designed.
Security
Messages can be encrypted to protect privacy and you can require users to authenticate
themselves before being allowed to receive messages. Security can be implemented using well-
known standards such as SSL or WS-SecureConversation.
Multiple Transports Encodings
Messages can be sent on any of several built-in transport protocols and encodings. The
most common protocol and encoding is to send text encoded SOAP messages using is the
HyperText Transfer Protocol (HTTP) for use on the World Wide Web. Alternatively, WCF
allows you to send messages over TCP, named pipes, or MSMQ. These messages can be
encoded as text or using an optimized binary format. Binary data can be sent efficiently using the
MTOM standard. If none of the provided transports or encodings suit your needs you can create
your own custom transport or encoding.
Reliable and Queued Messages
WCF supports reliable message exchange using reliable sessions implemented over WS-
Reliable Messaging and using MSMQ.
Durable Messages
A durable message is one that is never lost due to a disruption in the communication. The
messages in a durable message pattern are always saved to a database. If a disruption occurs, the
database allows you to resume the message exchange when the connection is restored. You can
also create a durable message using the Windows Workflow Foundation (WF).
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 11
Transactions
WCF also supports transactions using one of three transaction models: WS-
AtomicTransactions, the APIs in the System.Transactions namespace, and Microsoft Distributed
Transaction Coordinator.
AJAX and REST Support
REST is an example of an evolving Web 2.0 technology. WCF can be configured to
process "plain" XML data that is not wrapped in a SOAP envelope. WCF can also be extended to
support specific XML formats, such as ATOM (a popular RSS standard), and even non-XML
formats, such as JavaScript Object Notation (JSON).
Extensibility
The WCF architecture has a number of extensibility points. If extra capability is required,
there are a number of entry points that allow you to customize the behavior of a service.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 12
Architecture
The Windows Communication Service stands at the center of the software platform hosted
in Windows Azure. It processes HTTP requests from end devices, handles Workflow instances
and also communicates with the Teacher Interface WebApp built in ASP.NET MVC 3. (Fig 1)
Fig. 1 Basic Software Architecture
The connection between the WCF service and the End-devices and the ASP MVC3 webapp
is done through endpoints. The endpoints of a WCF Service are exposing its contract. So an
endpoint has three main properties:
An address(URL for accessing the endpoint)
Binding property defining how the data will be transferred( specifying protocols such as
SOAP over HTTP, SOAP over TCP and SOAP over Message Queues)
Contract
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 13
Implementation
In Windows Communication Foundation all services expose contracts. Contracts describes
what the service does. By applying the [ServiceContract] attribute on an interface, it will be
exposed as a public service contract and visible to WCF clients. It is equivalent with the Java
Remote interface.
Example: using System; using System.Collections.Generic; using System.Linq; using System.Runtime.Serialization; using System.ServiceModel; using System.ServiceModel.Web; using System.Text; namespace LiveXService {
[ServiceContract] publicinterfaceILiveXService {
[OperationContract] ICollection<string> GetClassrooms(int teacherId); [OperationContract] ICollection<string> GetLessons(string classroomName); [OperationContract] ICollection<string> GetExperiments(string lessonName); [OperationContract] void SelectLesson(string lessonName); } //some more methods
} }
Also to expose methods to clients, the OperationContractAttribute must be explicitly
applied. The OperationContract can only be applied on methods. The methods on the interface
that are not marked as OperationContracts are not visible to the WCF clients and accordingly
can‘t be consumed by them. This is the way we defined the service interface exposed for the
clients.
To implement the behavior the IliveXService interface must be implemented and each
method marked as [ServiceBehavior].
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 14
Notice also the InstanceContextMode.PerSession tag that provide users an instance lifetime
equal to the session lifetime. So for example, an instance created by the first call will be
available in the second call.
using System.Runtime.Serialization; using System.ServiceModel; using System.ServiceModel.Web; using System.Text; using System.Activities; using System.Activities.XamlIntegration; using System.IO; using CubeDevices; namespace LiveXService { ///<summary> /// Learning platform service ///</summary> [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession)] publicclassLiveXWCFService : ILiveXService { ///</summary> ///<param name="teacherId"></param> ///<returns>a collection of classroom names</returns> publicICollection<string> GetClassrooms(int teacherId) { //Fetch classrooms from database by teacherId SchoolDBDataContextdb = new SchoolDBDataContext(); var classrooms = from classroom in db.Classrooms select classroom; return matchingCustomers.ToList();
} } }
Fig. 2 - Entity class diagram, school db
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 15
ASP.NET MVC3 Web App
1.Introduction
―ASP.NET MVC is a web development framework from Microsoft that combines the
effectiveness and tidiness of model-view-controller (MVC) architecture, the most up-to-date
ideas and techniques from agile development, and the best parts of the existing ASP.NET
platform. It‘s a complete alternative to ―traditional‖ ASP.NET Web Forms, delivering
considerable advantages for all but the most trivial of web development projects.‖[2]
ASP.NET MVC web development platform was presented first in October 2007 at the very
first ALT.NET conference in Austin, Texas. The main benefit of ASP.NET MVC is the
separation of concerns through using the Model View Controller design pattern.
This pattern maps very naturally to web application platforms because usually web
applications combine several different technologies like executable code, HTML, databases and
through the MVC pattern, we can separate them in three main parts.
A usual client interaction using ASP MVC begins with the controller receiving the request.
The controller than modifies or interrogates the model and send back to the user and updated
view as response.
1.1 Model-View-Controller Overview
―The Model-View-Controller architectural pattern (MVC) divides an interactive application
into three components. The model contains the core functionality and data. Views display
information to the user. Controllers handle user input. Views and controllers together comprise
the user interface. A change-propagation mechanism ensures consistency between the user
interface and the model.‖[3]
Fig 3. MVC architecture for Web
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 16
1.2 Testability
Because through the MVC pattern we obtained a separation of concerns the application is
very easy maintainable and testable. So it is very easy to make Unit tests for each controller
using mock implementation and achieve automated testing
For testing we used the Nunit unit test tool.
1.3 Development tools
We used the following relevant development tools required by ASP.NETMVC:
Windows 7 Professional
Visual Studio 2010 Professional with SP1
SQL Server 2008 Express
.NET Framework 4
Controllers
Controllers are simple C# classes derived from the System.Web.Mvc.Controller base class
built in in the framework, responsible for handling incoming requests. The methods of the
controllers are called action methods and can be invoked via URL form the Web.
1.4 HomeController
The HomeController gets the GET request from the client browser and renders the action‘s
default view under ~Views.Home/Index.aspx
using System; using System.Collections.Generic; using System.Linq; using System.Web; using System.Web.Mvc; namespace LiveXTeacherApp.Controllers { publicclassHomeController : Controller { // // GET: /Home/ publicActionResult Index() { return View(); }
} }
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 17
1.5 Account Controller
The Account controller has two action results Sign In and Sign Out. The SignIn method
redirects to the Home Screen where a JavaScript will get the Identity providers from the cloud
and display them for the user to choose. We provide just login though the Windows Live
account.
To sign out we use a Microsoft.IdentityModel.Web.WSFederationAuthenticationModule
instance invoking the SignOut method. namespace LiveXTeacherApp.Controllers { publicclassAccountController : Controller { privateHrdClient hrdClient; public AccountController(HrdClient client) { hrdClient = client; } public AccountController() : this(newHrdClient()) { } // // This is the endpoint where the WS-Federation message will be posted. We are disabling the validation // here because we are expecting the form to have the xml WS-Federation message in it. // // POST: /Account/SignIn [HttpPost] [ValidateInput(false)] publicActionResult SignIn(FormCollection forms) { // We use return url as context string returnUrl = GetUrlFromContext(forms); // If there is a return URL, Redirect to it. Otherwise, Redirect to Home. if (!string.IsNullOrEmpty(returnUrl)) { return Redirect(returnUrl); } else { return RedirectToAction("Index", "Home"); } } //
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 18
// GET: /Account/SignOut [HttpGet] publicActionResult SignOut() { WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule; try { FormsAuthentication.SignOut(); } finally { fam.SignOut(true); } // Return to home after LogOff return RedirectToAction("Index", "Home");
}
///<summary> /// Gets from the form the context ///</summary> ///<param name="form"></param> ///<returns></returns> privatestaticstring GetUrlFromContext(FormCollection form) { WSFederationMessage message = WSFederationMessage.CreateFromNameValueCollection(newUri("http://www.notused.com"), form); return (message != null ? message.Context : null); }
Views
1.6 _Layout view
The Layout View contains the HTML for rendering the main screen <body> <divclass="page"> <divid='header"'> <divid="signindisplay"> @Html.Partial("_SignInPartial") </div> <h1> LiveX Teacher Interface</h1> <divclass="mango">
Defining a metro live tile: <divid="tile1"class="live-tile myClass"data-stops="100%"data-delay="7500"> <spanclass="tile-title">tile title</span>
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 19
<div> <ahref="http://www.google.com"target="_blank"> <imgclass="full"src=" /images/1pixel.gif"alt="1"/> </a> </div> <div> <pclass="metroBig">Schedule</p> </div> </div> </div> Footer bar with home button, twitter, facebook and theme selection: <footerclass="appbar"> <nav> <ulid="menu"class="appbar-buttons"> <li> <ahref="#"class="home"><imgsrc="/images/1pixel.gif"alt="home"/></a> <spanclass="charm-title">Home</span> </li> <li> <ahref="#"class="about"><imgsrc="/images/1pixel.gif"alt="about"/></a> <spanclass="charm-title">About</span> </li> <li> <ahref="#"target="_blank"class="twitter"><imgsrc="/images/1pixel.gif"alt="twitter"/></a> <spanclass="charm-title">Twitter</span> </li> </ul> </nav> <aclass="etc">•••</a> <ulclass="base-theme-options clear"> <li> <aclass="accent light"href="javascript:;"></a> </li> <li> <aclass="accent dark"href="javascript:;"></a> </li> </ul> </footer> </div> <divid="main"> @RenderBody() </div> </div> </body>
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 20
1.7 _IdentityProvidersWithServerSideCode
Identity provider list for selecting the desired login method( LiveID, Facebook, Google,
etc.)
<divid="identity-provider-content"> <h2> Sign in using one of the following Identity Providers </h2> <divid="identity-providers"> @foreach (HrdIdentityProvider identityProvider in Model) { <divclass="identity-provider"><buttonid="@Html.Raw(identityProvider.LoginUrl)">@identityProvider.Name</button></div> } </div> </div>
Access Control Service (Windows Azure AppFabric’s ACS)
The Access Control service (ACS) is the new solution for authenticating
REST(Representational State Transfer).
5.1 Technology used
Microsoft Visual Studio 2010
Windows Identity Foundation
Windows Identity Foundation SDK
Active Windows Azure account
5.2What Is the ACS?
According to [3]:„Simply put, the ACS is a standards-based token issuer in the cloud: a
hosted, multi-tenant offering available to every AppFabric account. Once you've signed up for a
Windows Azure Platform account, you can begin configuring your applications to rely on the
ACS to easily authorize access for trusted applications, users, and partners. Figure 1 illustrates a
very high-level perspective on the flow of communication among participants. The client
application (Service Consumer) requests a token from the ACS—supplying either a symmetric
key or Security Assertion Markup Language (SAML) token. The ACS (Token Issuer) verifies
that the request came from a trusted caller; evaluates rules associated with the scope of the
request; and issues a security token carrying the claims granted the caller. The web resource
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 21
(Service Provider) verifies that the security token was issued by a trusted token issuer (the ACS
account) and uses the issued claims to authorize access accordingly.‖
Fig. 4 ACS AuthorizationCommunication Flow
a) The client ( browser ) requests a page from the RP(Relying Partner) – ASP.NET MVC
3Web Page
b) Since the request is not yet authenticated, the RP redirects the user to the authority that
it trusts, which is ACS. The ACS presents the user with the choice of IPs that were specified for
this RP. The user selects the appropriate IP.
c) The client browses to the IP's authentication page, and prompts the user to log on.
d) After the client is authenticated (for example, the identity credentials are entered), the IP
issues a security token.
e) After issuing a security token, the IP redirects the client to ACS and the client sends the
security token issued by the IP to ACS.
f) ACS validates the security token issued by the IP, inputs the identity claims in this
token into the ACS rules engine, calculates the output identity claims, and issues a new security
token that contains these output claims.
g) ACS redirects the client to the RP. The client sends the new security token issued by
ACS to the RP. The RP validates the signature on the security token issued by ACS, validates the
claims in this token, and returns the page that was originally requested.
5.2 Configure Windows Azure AppFabric Access Control Service
The Access Control Service requires an active Windows Azure account. After logging in to
the Windows Azure Management Platform certain steps need to be follows to configure different
Identity Providers to be used
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 22
Windows Azure Management Portal:
To begin using Access Control Service (ACS) in Windows Azure, an ACS namespace must
be created as shown in the figure below(LiveXac). The namespace provides a unique scope for
addressing ACS resources from the ASP.NET MVC 3 Website.
Fig. 5 : Azure Management portal
Windows LiveID Identity Provider in Windows Azure:
Fig. 6: LiveID Identity provider
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 23
Windows Live Login Client
After creating an ASC namespace in Windows Azure, adding Identity Service Providers (
Windows Live ID) and Relying Party Application ( ASP.NET MVC 3) an STS(Secure Token
Service) reference must be added to the ASP.NET MVC 3 Project and add the WS-Federation
Metadata URI obtained from Windows Azure Identity Service Management Portal.
This way a reference will be added to the Microsoft.IdentityModel.dll assembly and the
Web.config file will be updated to hold the configuration of the Windows Identity Foundation
(WIF) in the ASP.NET MVC 3 webapp.
The _LayoutView includes through the Razor notation @Html.Partial the Login PartialView
defined in _SignInPartial.
<body> <divclass="page"> <divid='header"'> <divid="signindisplay"> @Html.Partial("_SignInPartial") </div> </div> //more html </div> </body>
The _SignInPartial checks if the user is authenticated. If this is true, then the SignOut
ActionLink is displayed, else the Account Controller is invoked using the action result
IdentityProvidersWithClientSideCode.
@if (Request.IsAuthenticated) { <div> <text>Welcome <b>@Context.User.Identity.Name</b>! [ @Html.ActionLink("Sign Out", "SignOut", "Account") ]</text> </div> } else { @Html.Action("IdentityProvidersWithClientSideCode", "Account"); }
The IdentityProvidersWithClientSideCodefrom the AccountController represents an
attribute that is used to indicate that an action method should be called only as a child action.
The ChildActionOnly attribute is used to indicate that an action method should be called
only as a child action.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 24
[ChildActionOnly] publicPartialViewResult IdentityProvidersWithClientSideCode() { WSFederationAuthenticationModule fam = FederatedAuthentication.WSFederationAuthenticationModule; HrdRequest request = newHrdRequest(fam.Issuer, fam.Realm, context: Request.Url.AbsoluteUri); return PartialView("_IdentityProvidersWithClientSideCode", request); }
The WSFederationAuthenticationModule instance gets the Azure appFabric Access Control
Service (ACS) authentication properties from the Web.config file. The Security Token
Service(STS) WS-Federation metadata document location is:
https://LiveXac.accesscontrol.windows.net/ (configured previously in Windows Azure
Management Portal).
Next, the _IdentityProvidersWithClientSideCode View is invoked which contains a
Javascript for getting the Identity providers from the cloud using Ajax.
@*Using client side script to get the HRD data from ACS*@ <scripttype="text/javascript"> $(function () { $("#signIn").click(function () { // // Explicit JSONP callback can be used to do client side caching of identity provider data. // $.ajax({ url: "@Html.Raw(Model.GetUrlWithQueryString())", dataType: "jsonp", success: function (data, textStatus, jqXHR) { $("#main").hide("blind", 500, function () { // Clear the main content $("#main").empty(); $('<div id="identity-provider-content">' + '<h2>Sign in using one of the following Identity Providers</h2>' + '<div id="identity-providers"></div></div>').appendTo("#main"); for (var i in data) { $(GetIdentityProviderTag(data[i], i)).appendTo("#identity-providers"); $("#button" + i + " button").data("signInUrl", data[i].LoginUrl); break } // // make buttons jqueryui buttons // $("button").button();
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 25
// // add click behaviour // $("button").click(function () { window.location = $.data(this, "signInUrl"); }); }); $("#main").show("blind", 750); } }); // // Stop event bubbling // returnfalse; }); }); // Gets the div tag and its content for an identity provider function GetIdentityProviderTag(idp, index) { return'<div class="identity-provider" id="button' + index + '"><button>' + idp.Name + ' </button></div>'; } </script> <div> <ahref=""id="signIn">Sign In</a> </div>
User Interface
1.8 Design and Graphics
We designed the user interface following the Metro UI design principles.A clear and
straightforward definition can be found in [MUSM05]:
“The Metro design language was designed specifically to consolidate groups of common
tasks to speed up usage. This is accomplished by excluding superfluous graphics and instead
relying on the actual content to also function as the main UI. The resulting interfaces favor
larger hubs over smaller buttons and often feature laterally scrolling canvases. Page titles are
usually large and consequently take advantage of lateral scrolling.
Animation plays a large part, with transitions, and user interactions such as presses or
swipes recommended to always be acknowledged by some form of natural animation or motion.
This is intended to give the user the impression that the UI is "alive" and responsive, with "an
added sense of depth.”
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 26
Fig. 7 Windows 8 Start Screen
1.9 Implementation
For the user interface, we used the following:
jquery-1.7.2.js
CSS 2.1
Metro.js JavaScript plug-in for jQuery
The interface is composed of metro style live tiles, each giving a different precise and
intuitive user experience.
We have thought about tiles for: Class management (students), Experiment designer, Lesson
scheduler and Experiment Marketplace.
Fig. 10 LiveX Teacher Interface
Live tile HTML example:
<divid="tile2"class="live-tile blue"data-direction="horizontal"data-stops="23px,50%,75%,100%,50%,0"> <spanclass="tile-title">tile title</span> <div>
The new Windows 8 Start Screen, making use of flat, colored live tiles and a laterally scrolling canvas as used in Windows Phone and Xbox 360 Dashboard
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 27
<imgclsas="full"src="/images/1pixel.gif"alt="1"/> </div> <divclass="myClass"> <p>This tile swaps the front image at each stop and fades the transition</p> </div> </div>
This tile will animate through the following JavaScript script: <scripttype="text/javascript">
//flip-list: image swapps and triggerDelay //an array of 9 images to swap between var imgs = [{ src: 'Images/Chrysanthemum.jpg', alt: '1' }, { src: '/Images/metroIcons_light.jpg', alt: '2' }, { src: 'Images/Hydrangeas.jpg', alt: '3' }, { src: 'Images/Jellyfish.jpg', alt: '4' }, { src: 'Images/Koala.jpg', alt: '5' }, { src: 'Images/Lighthouse.jpg', alt: '6' }, { src: 'Images/Penguins.jpg', alt: '7' }, { src: 'Images/Tulips.jpg', alt: '8' }, { src: 'http://www.drewgreenwell.com/content/images/sample/9tw.gif', alt: '9' } ];
$("#tile2").liveTile( { mode: 'slide', frontImages: imgs, //choose images randomly from the array frontIsRandom: true, fadeSlideSwap: true } ); </script>
The design and look and feel are defined in the Site.css file. Tiles properties: #tiles { overflow: hidden; position: relative; max-width: 200px; float:left; margin:0px; padding-bottom:100px; }
Tile text title properties: .tile-title { font-family: SegoeUILight; color: #000000;
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 28
font-size: 36px!important; }
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 29
Educator Interface in Silverlight for Embedded
Silverlight for Windows Embedded
Silverlight for Windows Embedded is a native (C++ based) user
interface development framework for Windows Embedded Compact.
Silverlight for Windows Embedded is compatible with
Silverlight 3 XAML and provides a set of equivalent classes for
supported XAML elements. It is also compatible with existing
Windows Embedded Compact window controls, so existing window
controls can be used.
It is very different from the flavors of Silverlight for Windows because it has no managed
API, no browser plug-in and is out of browser only.
XAML(eXtensible Application Markup Language) is an XML language for expressing a
user interface and presentation layer including controls, animation effects. The key thing is that it
is a declarative markup and in the XML, real objects are represented. So when the runtime parses
the XML and represents the UI on the screen for the users to interact with, all the controls like
Grid, Textbox are represented as objects:
In .NET code behind:
The problem with Compact framework in the past was that it was windows forms space
Silverlight 3 XAML has many built-in controls (customizable styles and templates), rich
presentation capabilities (solid, gradient, image brushes, opacity, pixel effects such as blur and
drop shadow) and animations (movement, visibility, color, change properties of controls over
time)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 30
Windows Embedded Silverlight Tool with Microsoft Expression
Blend
Microsoft Expression Blend 3 is a very good tool for
design visually user interfaces for embedded platform
(Silverlight for Windows Embedded Application project)
Click, Load and other events for the user interface
controls must be hooked up, but the generated C# code
must be ignored because the code behind will be
implemented in native C++ code.
Fig. 11: Microsoft Expression Blend
Next, the generated Expression Blend project must be imported in a Silverlight for Windows
Embedded application in Visual Studio.
All the Silverlight UI initialization logic is implemented in App.class. A class gets generated
for each XAML page designed in Expression Blend (MainPage.xaml => MainPage.cpp) and also
event handler methods for all events defined in XAML.
Therefore, we could work on the XAML in Expression Blend and the code behind in Visual
Studio at the same time.
C++ code for locating a button in the object tree resulted by parsing the XAML and hooking
up a Click event handler:
// Locate the button in the object tree IXRButtonPtr pButton; pVisualHost->GetRootElement(&pRootElement); FindName(pRootElement, L”Button1”, IID_IAXButton, &pButton); // Add OnClick Event handler pButton->AddClickEventHandler(CreateDelegate(pEventHandler, &EventHandler::OnButtonClick)); //…
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 31
It is also possible to control the SWE UI from managed code by setting up callback
methods. Therefore, a set of required functions must be exported from the DLL so that C# code
can call them.
Fig. 12: Functionality of a Silverlight Application
Basically the needed functionalities are control functions to run the XAML UI, Get/Set
properties of controls and animation Controls.
All the exported functions must be declared in SWENative.h:
In C#, the functions can be declared using the DllImport attribute:
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 32
In the native event handler, we call the Callback function:
1. eBox 3310
As embedded hardware platform for the kindergarten
teachers we used the eBox 3310.
The eBox-3310A-JSK compact computing device with 1.0
GHz processor, 512MB RAM, Ethernet, Compact Flash slot,
Audio, 3 USB 2.0, 2 Serial port, VGA
Fig. 13 - eBox-3310A-MSJK front and
back view(Shown with optional Wi-Fi)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 33
2. User Interface in Silverlight for Windows Embedded
To follow the overall progress of all children and set up the games for the IQubes we
developed a very nice and intuitive user interface.
The software runs on the eBox3310 and the user can interact with it through a touch screen
to offer an intuitive and straightforward experience, without requiring anyprevious computer
experience or training.
First, the educators can log in with a username and a password. This allows the same eBox
to be used by different tutors/kindergarten classes, reducing the overall solution deployment
costs.
After some usability, studies we
concluded that an interface resembling a
notebook would fit the best (see Fig. 14).
Bookmarks on the left-hand side of
the notebook identify different children.
This allows the educator to browse
through the notebook very easy and
intuitive, without search boxes or list
boxes.
When a bookmark is selected, the notebook opens at the specific page through a very
smooth page flip animation.
For modeling the GUI, we used Silverlight for Windows Embedded, running on the new
Windows Embedded Compact 7 platform, which has Silverlight 3 XAML support.
Once we have designed the UI in an Expression Blend 3 project, we created a SWE
Application using the wizards and tools in Visual Studio 2008. The wizard creates a Platform
Builder subproject that provides a minimalist C++ application. The wizard also generates
wrapper controls for each of the User Controls that are designed in Blend.
To create an initial Silverlight for Windows Embedded application template from the
Expression Blend project, we used the Windows Embedded Silverlight Tools Platform Builder
Subproject Application Wizard.
The Subproject Application Wizard generated a Platform Builder subproject from the
resources and XAML files of the source Expression Blend project. After we incorporated this
Platform Builder subproject into our Windows Embedded Compact 7 OS design, we could
develop, test, and deploy our application on the eBox device.
Fig. 14 Interface created used
Silverlight for Windows Embedded
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 34
We chose Silverlight for Windows Embedded because it offers OEM developers and
designers the tools and platform ability to create attractive, intuitive and highly customizable
device UIs, has been optimized to ensure performance on devices with embedded processor and
memory requirements, accelerates device development by allowing UI developers to iterate on
device experiences without delaying device development engineers.
The separation of business logic and UI allows us to respond to market changes, provides
multiple UI concepts and makes updates through the development cycle without requiring
recreation of the underlying business logic.Compared to traditional embedded UI development,
the Silverlight for Windows Embeddedframework can provide OEMs with significant time to
market advantages while providing a level of UI richness new to embedded devices.
3. Encountered problems
During development process, we faced some problems and limitations:
First, we had a big problem with the deployment of the application with the eBox. If we
added controls like buttons, textboxes, list boxes and so on to the user interface, after
deployment, just the application label appears on the toolbar, but the application window was
transparent. This was very difficult to solve and it took us some weeks of debugging, after which
we realized that we had to set explicitly width and height to the user control. We did not declared
width and eight of the user control in XAML, because we expected the UI to fit automatically to
the screen size, as in Silverlight for Desktop Browser.
Fig. 15 Interface created used
Silverlight for Windows Embedded
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 35
Second, we encountered unexpected stop of the application with retcode 0 at deployment.
This happened because we defined the control template resources in a resource dictionary. After
we changed it and defined them in the current document, the error disappeared. Some small
synchronization problems between Blend and VS2008: When deleting some elements in Blend,
when updating the project in VS2008, the button pointers and the event handlers that have been
deleted in Blend must be deleted manually in Visual Studio.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 36
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 37
Windows Workflow Foundation interoperability with embedded devices
Abstract
A workflow is a ―set of relationships between all the activities in a project, from start to
finish. Activities are related by different types of trigger relation. Activities may be triggered by
external events or by other activities.‖[DFWK01] People inherently use workflows to describe
activities, processes, or decisional flows either by sequences of events or relationships or
transitional triggers, their application being limited not only to computer science.
This paper will describe the Windows Workflow Foundation technology by Microsoft,
which aims to merge the workflow concept with functional software.
Windows Workflow Foundation provides a programming model, in-process workflow
engine and rehosted designer to implement long-running processes as workflows within .NET
applications[WWFH02]. The framework currently provides three types of workflow models,
each suitable to various scenarios: sequential workflow, state machine and flowchart. Workflows
provide and unified approach to designing and maintaining applications, providing out of the box
activities, but also allowing developers to write their own custom activities, to encapsulate in
tangible and configurable entities.
Another motivation for using workflows is the possibility to dynamically compile and load
them at runtime, which is a used aspect in our software platform, allowing regular people to
make use of custom activities, built-in activities and our rehosted designer to easily create
workflows.
The other key aspect are the embedded devices, since the workflows are decoupled from the
devices a large number of end-devices is supported. This paper will focus on Windows Phone 7
as an end device.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 38
Introduction
Not all people may know the term ―workflow‖, yet we inherently see and make them
throughout our life, starting with general school, and many diagrams in science books list
diagrams as workflows.
In the business world, a workflow is used to list the sequential steps in a project, from start
to finish. Storyboards in a studio list the sequences of activities in a movie scene. In computer
science, state machines show transitions in hardware, datasheets list relationships between
components, high-level software design describes the process with flowcharts.
In short, we are surrounded by workflows, without actually knowing it.
In 2006 Microsoft introduced the Windows Workflow Foundation (WWF) as part of the
.NET framework version 3, with the aim to make applications ―as easy as possible to understand,
both for its creators and for the people who maintain it.‖ and ―a way to keep the application‘s
logic unified, making it more understandable, while still letting the application scale.‖. While
rough around the edges it provided a great basis, providing a unified style for applications,
opposed to various personal coding styles that currently exist.
The .NET 3.5 Framework brought some updates and provided communication with the
Windows Communication Foundation. While it provided some great capabilities for individual
developers, it did not provide any features for enterprise developers, who are the most to benefit
from it.
Six years later the technology is refined, more features are added and it was made even
easier to use. The .NET framework version 4 brings a completely re-engineered paradigm for
workflows aimed to aid all .NET developers, coupled with important performance
enhancements, which bring a ten-fold improvement in performance.
The Windows Workflow Foundation was created for ―long running applications‖, and to
scale well, so waiting for input is not something to be desired, instead mechanisms are
implemented to store the Workflow‘s current state and suspend it until it is needed.
Windows Phone 7 is a milestone in the smartphone market, being the first modern operating
system for such devices created by Microsoft, being built on the .NET platform and designed to
offer a great experience for the end-user, and as such being constructed for him. As with any new
technology, it is still developing and a lot of useful technology for developers, such as
Background Task, Multi-Tasking or support for the local storage database SQL compact was
only added later with the Windows Phone 7.5/7.1 codenamed Mango release, in September
2011. Still, the operating system has reached enough maturity to be a dangerous competitor to
the existing one, being able to offer a smoother experience using lower hardware specifications
than Android, and being less fragmented with developers only needing to support a small
number of configurations due to the tight guidelines issued by Microsoft. The latest iteration
even lowered the RAM requirement to 256 MB [HWWP03].With the use of workflows,
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 39
processing is done in the cloud, lowering the needed CPU power, while offering a negligible
drop in battery life due to the built-in power consumption capabilities.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 40
Theoretical aspects
Basic workflow aspects
At the core of a Workflow lies the basic block, called an activity. Many activities, such as
primitives that control the application flow (If, While, For), or error handling (Try/Catch) are
provided out of the box, however developers can also write their own blocks. These manifest
themselves as custom activities and for their user can be seen as a code activity, which is a non-
configurable block, or a custom activity with designer. The activities that have designers allow
developers to configure them as with the built-in activities from within the designer.
The base activity library provides a plethora of activities, these being the more basic ones
[WWAL04]:
Assign: assigns a value to a variable in the workflow.
Compensate: provides a way to do compensation, such as handling a problem that occurs
in a long-running transaction.
DoWhile: executes an activity, and then checks a condition. The activity will be executed
repeatedly as long as the condition is true.
Flowchart: groups together a set of activities that are executed sequentially, but also
allows control to return to an earlier step.
ForEach: executes an activity for each object in a collection.
If: creates a branch of execution.
Parallel: runs multiple activities at the same time.
Persist: explicitly requests the WF runtime to persist the workflow.
Pick: allows waiting for a set of events, then executing only the activity associated with
the first event to occur.
ReceiveMessage: receives a message via WCF.
SendMessage: sends a message via WCF.
Sequence: groups together a set of activities that are executed sequentially. Along with
acting as a workflow‘s outermost activity, Sequence is also useful inside workflows. For
example, a While activity can contain only one other activity. If that activity is Sequence, a
developer can execute an arbitrary number of activities within the while loop.
Switch: provides a multi-way branch of execution.
Throw: raises an exception.
TryCatch: allows creating a try/catch block to handle exceptions.
While: executes a single activity as long as a condition as true.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 41
with more being available, and more to come, either from CodePlex or third party developers.
These activities can be combined to create the Workflow (WF).
The initial version of the WWF had two types of workflows: sequential workflow and state
machine. In 3.5, this was extended to three by adding the flowchart. Version 4 initially removed
state machine, only to bring it back with an update due to popular request, even though Microsoft
initially stated that the Flowchart workflow should cover the lack of state machines [WWKF05].
To extrapolate, it means that there currently are three types of workflows that can be used in
the Windows Workflow Foundation:
● Sequential Workflow
● Flowchart
● State Machine.
The Sequential Workflow lists a set of steps that are executed sequentially. It allows the use
of activities control such as If and While, but can only move forward, there is no way to return to
an activity executed earlier in the control flow.
The Flowchart works in much the same way, but with the possibility to return to a previous
step in the control flow, emulating real processes much better.
The State Machine provides the possibility to create finite state machines (FSM), abstract
machines consisting of a set of states (including the initial state), a set of input events, a set of
output events, and a state transition function. The function takes the current state and an input
event and returns the new set of output events and the next state. Some states may be designated
as "terminal states". The state machine can also be viewed as a function which maps an ordered
sequence of input events into a corresponding sequence of (sets of) output events. State
Machines provide a basis for long running applications, since the transition of states is triggered
externally and a set of activities is executed when the state is entered or exited. For the rest of the
time the state can be stored, and the workflow is suspended, releasing CPU cycles for other
applications.
These workflows are then loaded and executed by the Workflow Engine, versions up to
.NET 3.5 used the same engine for all workflows, while .NET 4 instantiates an engine instance
for each workflow, making it possible to isolate individual workflows and run them in parallel,
creating the basis for scalable applications.
From the underlying perspective, a workflow can be described twofold:
● by creating a XAML file (essentially derived from an XML type file), which can
be created using the designer, either a rehosted version, or the one built into Visual
Studio 2010, or by directly writing XML in a text editor
● Programmatically, by using C# or VB .NET.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 42
While the programmatic version is assembled into a class-like entity, which is instantiated
as an object and executed by the WorkflowInvoker, XAML files can be dynamically loaded from
a stream, compiled and executed. This makes the Workflow Engine a platform able to
dynamically run different versions or variants of the same workflow, providing a way to modify
the logic of the application without downtime or deployment time. Since at this point the XAML
is just a file, it can also be stored on servers, decoupling the logic part from the executor part.
To illustrate a simple sequential workflow, in all instances, suppose one is created to
perform a simple mathematical operation: c = a + b, where a and b are numbers which are
dynamically given.
In plain C# the code would look something like this:
Listing 1: Simple C# method
int add(int a, int b) { int c; c = a + b; return c; }
This function would then be called from another c# method, like this:
result = add(3,5);
The designer allows via drag-and-drop to place blocks and customize them. It also allows
the placement of In and Out arguments. In this case the Assign primitive is needed, a and b are In
arguments and result is an Out argument.
This is the visual representation, created by the use of the design tools provided in Visual
Studio.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 43
Figure 16: A Simple sequential Workflow
The underlying XAML, which is generated based on the design, looks like this.
Listing 1: Underlying XAML code
<Activity mc:Ignorable="sap" x:Class="WorkflowConsoleApplication.Add"> <x:Members> <x:Property Name="a" Type="InArgument(x:Int32)"> <x:Property.Attributes> <RequiredArgumentAttribute /> </x:Property.Attributes> </x:Property> <x:Property Name="b" Type="InArgument(x:Int32)"> <x:Property.Attributes> <RequiredArgumentAttribute /> </x:Property.Attributes> </x:Property> <x:Property Name="result" Type="OutArgument(x:Int32)"> <x:Property.Attributes> <RequiredArgumentAttribute /> </x:Property.Attributes> </x:Property> </x:Members> <sap:VirtualizedContainerService.HintSize>304,240</sap:VirtualizedContainerService.HintSize> <mva:VisualBasic.Settings /> <Sequence DisplayName="add" sad:XamlDebuggerXmlReader.FileName="Add.xaml" sap:VirtualizedContainerService.HintSize="264,200"> <sap:WorkflowViewStateService.ViewState> <scg3:Dictionary x:TypeArguments="x:String, x:Object"> <x:Boolean x:Key="IsExpanded">True</x:Boolean> </scg3:Dictionary> </sap:WorkflowViewStateService.ViewState>
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 44
<Assign sap:VirtualizedContainerService.HintSize="242,58"> <Assign.To> <OutArgument x:TypeArguments="x:Int32">[result]</OutArgument> </Assign.To> <Assign.Value> <InArgument x:TypeArguments="x:Int32">[a + b]</InArgument> </Assign.Value> </Assign> </Sequence> </Activity>
This can be typed by hand in a regular text editor if desired. As observed, it defines the used
In/Out Arguments, the Activities inside as well as the default viewing style. This means that an
activity that was viewed in a certain way when saved will be visible in that same way when
reopened.
A programmatic representation, encapsulated with an initializer from a static method would
look like this:
Listing 2: Programmatic simple sequential activity
static Activity add() { InArgument<int> a = new InArgument<int>();
InArgument<int> b = new InArgument<int>(); OutArgument<int> result = new OutArgument<int>();
return new Sequence {
Activities = { new Assign<int> { To = new OutArgument<int>((env) => result.Get(env)), Value = new InArgument<int>((env) => a.Get(env) + b.Get(env)) } }
}; }
Calling the workflow is a little less straightforward. First, let us assume the workflow is
defined as a class like entity. This means you can instantiate the workflow. Afterwards a
dictionary has to be built, mapping the In/Out arguments to variables. Next, the Workflow will
be invoked using the WorkflowInvoker and the results will be extracted from the output.
Listing 3: Instantiation and execution of a workflow with arguments
IDictionary<string, Int32> inputs = new Dictionary<string, Int32> { {"a", a}, {"b", b}, };
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 45
IDictionary<string, Int32> outputs = WorkflowInvoker.Invoke(new Add(), inputs); result = outputs["result"];
If the workflow is stored in XAML form, the XAML has to be first loaded and then
compiled and executed. This can be done in one single line of code:
WorkflowInvoker.Invoke(ActivityXamlServices.Load("Add.xaml"), inputs);
Handling Exceptions
Since at the core a workflow runs like a .NET application, method calls that generate errors
will throw exceptions that should be caught. To facilitate this, the TryCatch and Throw activities
are provided. These work similarly to their pure C# counterparts, and in the designer look like
this. The throw activity has no designer interface; the name and type of the exception to be
thrown are specified in the Properties tab in the designer.
try { } catch (System.Exception ex) { } finally { }
throw new System.Exception();
Figure 17: TryCatch and Throw Activities next to the C# counterparts
Custom Activities
The power of the Workflows lie in the custom activities, which are custom functionalities
encapsulated in blocks. These blocks may have a designer part written using Windows
Presentation Foundation (WPF) to provide aid in configuring them.
Activities are designed as classes and implemented as CLR types, which extend existing
ones. The most common ones derive from NativeActivity since they interact with the Workflow
runtime. The executing code is written by overriding theExecute(NativeActivityContext
context)method. There is also the possibility to create a designer.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 46
Figure 18: Activity Type Hierarchy
This allows a custom activity to have user interface items such as buttons, lists, and
checkboxes in the Workflow Editor, allowing a better usage experience.
Persistence and Storage
Workflow persistence refers to the durable snapshot of a workflow, which provides a fast
recovery method in case of power loss, or the possibility to store the state and unload the
workflow to increase efficiency, scalability and resource usage. The .NET Framework version 4
ships with the SqlWorkflowInstanceStoreclass, which allows persistence of data and metadata
about workflow instances into a SQL Server 2005 or SQL Server 2008 database, as well as with
SQL Azure.
To use persistence the developer has to explicitly enable it by associating the Workflow
with an instance store. Once this is done, there are several predefined persistence points where
the state is automatically saved.
These are [WWKF05]:
● When a TransactionScope activity completes or a TransactedReceiveScope
activity completes.
● When a workflow instance becomes idle and the WorkflowIdleBehavioris set on
workflow host. This occurs, for example, when you use messaging activities or a Delay
activity.
● When a WorkflowApplication becomes idle and the PersistableIdle property of
the application is set to PersistableIdleAction.Persist.
● When a host application is instructed to persist or unload a workflow instance.
● When a workflow instance is terminated or finishes.
● When a Persist activity executes.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 47
● When an instance of a workflow developed using a previous version of Windows
Workflow Foundation encounters a persistence point during interoperable execution.
Workflow Tracking and Tracing
The workflow tracking and tracing features in .NET 4 were built from the ground up and are
meant to provide visibility into the workflow execution by transparently instrumenting a
workflow to emit records during key events. The infrastructure for this feature is available to all
.NET 4 Framework Workflows; the only action on the part of the developer is to configure the
amount of stored data. Tracking can be used to collect business analytical data, in addition to the
diagnostic data. The other component is Tracing. Tracing is primarily a diagnostic capability,
however both use the ETW (Event Tracing for Windows) system to store events in the Windows
Log. While .NET 3.5 offered an SQL Tracking participant out of the box, it was cumbersome
and logged more than needed. This tracking participant is removed in .NET 4. However WCF
and WF tracking to an SQL database can be done with the use of the ―Dublin‖ Application
Server. When using persistence in conjunction with tracking, Microsoft recommends using the
same Database for both to decrease overhead.
There are three primary components at play here:
Tracking Records: which are objects that describe the event
Tracking Participants: which are objects that subscribe to tracking records and
hold the logic to process them (eg. Write to a File, Write to a Database).
Tracking Profile: which are objects designed to filter out unneeded events issued
by the Workflow Instance and work in conjunction with the Tracking Participant.
The infrastructure follows a publish-and-subscribe model. The Workflow Instances are the
publishers of the Tracking Records while the Tracking Participants are subscribers which register
as extensions of the Workflow. The High-level view of the mechanism is listed in this figure:
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 48
Figure 19: Architecture of the Tracking Mechanism
Rehosted designer
The rehosted designer is the final key aspect, which is used in our solution. Essentially, it
provides the mechanism to distribute the core of an editor similar to the one integrated in Visual
Studio with client applications, be it a Desktop application or a Web Application.
Both are WPF applications and as such follow the MVVM(Model View ViewModel)
pattern.
As stated by [IMVW09], Model/View/ViewModel is a variation of Model/View/Controller
(MVC) that is tailored for modern UI development platforms where the View is the
responsibility of a designer rather than a classic developer. The designer is generally a more
graphical, artistic focused person, and does less classic coding than a traditional developer.. The
design is almost always done in a declarative form like HTML or XAML, and very often using a
WYSIWYG tool such as Dreamweaver, Flash or Sparkle. In short, the UI part of the application
is being developed using different tools, languages and by a different person than is the business
logic or data backend. Model/View/ViewModel is thus a refinement of MVC that evolves it
from its Smalltalk origins where the entire application was built using one environment and
language, into the very familiar modern environment of Web and now Avalon development.
This new pattern attempts to fully separate the Model, creating a ViewModel which is
responsible for interacting with the data and update of the user interface, providing data types
which bind directly to the controls.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 49
Windows Phone 7 software development
Applications for Windows Phone 7 are written in C# and the technology used is .NET
Framework Compact coupled with Silverlight. There are some inherit limitations to the C# built
into Windows Phone 7 due to the resource limitations of the devices. As such, some assemblies
are incomplete, and Reflection.Emit is missing. Reflection is useful if you want to develop
applications to create and load types dynamically at runtime, which as we will see later is
something we wanted to do.
Microsoft delivers a great toolchain including an emulator, which tries to help developers
create applications quickly. The integration is greater than with Android and its IDE of choice –
Eclipse, allowing quick debugging of the applications.
Figure 20: Windows Phone emulator running test software (portrait app left – panorama app right)
Windows Phone applications have to respect some design guidelines, which Microsoft
calls metro, as such applications have a similar look and feel with identical fonts and transition
animations. Applications are using the MVVM (Model View ViewModel) pattern, already
discussed under the Rehosted Designer chapter, which is crucial for a WP applications where the
number of threads is limited and the illusion of parallelism is offered through asynchronous
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 50
operations via events.Windows Phone applications have a main class called App, residing in
App.xaml/App.xaml.cs and a Mainpage.xaml/Mainpage.xaml.cs class, which holds the main
user interface.
There are various templates offered, which again, follow the Metro design guidelines,
such as Portrait Page and Panorama Page (scrolling page).
Another aspect in Windows Phone 7 is the isolated storage (which also holds the database).
This is a per application, private, user storage space. This means, only the application itself can
see the contents and work with it. Interacting with it is done like with regular desktop
applications, the concept of Folders and Files is present. The following listing shows how to save
a file from an Image object to the isolated storage
Listing 4: Saving a file to the isolated storage
IsolatedStorageFileStreamfileStream = isoStorage.CreateFile(filepath); StreamResourceInfosri = null; Uri uri = new Uri(filepath, UriKind.Relative); sri = Application.GetResourceStream(uri); BitmapImage bitmap = new BitmapImage(); bitmap.SetSource(sri.Stream); WriteableBitmapwb = new WriteableBitmap(bitmap); Extensions.SaveJpeg(wb, fileStream, wb.PixelWidth, wb.PixelHeight, 0, 92); fileStream.Close();
The database is stored inside asdf file (SQL compact) and the only way to access it in a
reasonable is with a LINQ query. LINQ or Language Integrated Query is a language developed
by Microsoft to interrogate data resources. What‘s interesting is that it can be used with lists as
well as with the database itself. The following is an example of a LINQ query
Listing 5: LINQ query example
varexp = from a in App.Database.GetTable<ExperimentData>() wherea.ExperimentId == experimentID select a;
It is equivalent to the T-SQL query: Select * from ExperimentData where ExperimentId == experimentID
What‘s interesting is that the LINQ queries are actually translated automatically in the
background to T-SQL queries, however there is no way to work with T-SQL directly.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 51
Sensors
Being an embedded device Windows Phone 7 has lots of sensors that interact with the
environment, however some are not directly user exploitable. The list of sensors currently
supported to third party developers is[HWWP03]:
Standard Hardware
Accelerometer
A-GPS
Optional Hardware
Compass
Gyro
Primary Camera
Front-facing Camera
Interfacing which such sensors is not a problem. The resulting values are given in natural
units, such as the accelerometer returning the number of G force units on each axis, the GPS
returning the coordinates and the cameras returning an Image object. However, there is no way
to bypass the virtual machine and directly hook to the raw data, or to access other sensors such as
the ambient light sensor.
IronRuby
―IronRuby is an open-source implementation of the Ruby programming language which is
tightly integrated with the .NET Framework. IronRuby can use the .NET Framework and Ruby
libraries, and other .NET languages can use Ruby code just as easily‖[BWFC07].
What this means is that we can use IronRuby to dynamically load a script from a simple
source file (or stream, string, database entry) and interpret it dynamically at runtime. We are
using this to bypass the limitation of the .NET Compact C# implementation. While this sounds
good on paper, we quickly found out that it is not possible to write the whole program in C# and
that implementing too much in IronRuby, such as directly registering Ruby methods to event
handlers would cause it to crash. So we resumed ourselves to have Ruby register C# methods
that call inside them the Ruby method. For example, a simple implementation of the on_click
handler in Ruby:
button1 = Button.new button1.click do |sender, args| MessageBox.show("Hello World!") End
However, this didn‘t work, the end result being a mix of C# and IronRuby, in C#:
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 52
public void monitor_control(object but, string funct) { Button button = (Button)but; button.Click += delegate(object sender, RoutedEventArgs e) { this.Engine.Operations.InvokeMember(this.ScrScope, funct); }; }
And in Ruby:
button1 = Button.new Host.children.add(button1) Host.monitor_control(button1, "button_listener") def button_listener MessageBox.show("Hello World!") update_text end
This works flawlessly, and essentially achieves the same effect as the simple method above,
however it is not so straightforward.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 53
Implementing the workflows
The Workflow Engine is used inside the LiveX Learning Platform to provide the algorithm
support for the High Level Business logic. The reasoning behind this is to strip away the
calculation aspect of the software from the end devices and move it to the cloud. This allows the
creation of generic thin clients on devices, which require minor to no updates, lower the system
requirements, and provide consistent business data to relevant parties.
Workflows are also useful to control the flow of the educational games since some steps are
triggered asynchronously at various times, thus using the persistence mechanism coupled with
state machines allows for a better resource management increasing scalability, which is crucial
when dealing with a large number of simultaneous clients.
Finally yet importantly, since the workflow is a generic concept, the logic can be modeled
by persons who are not proficient at writing code, thus a Workflow Editor can be distributed
with the software, allowing teachers to develop experiments.
The workflow engine is hosted in the cloud so communication with the end-devices has to
be done through the Windows Communication Foundation; however, it only acts as a messenger,
relaying the information.
The software platform uses two types of workflow types, the simple sequential workflows
define simple processing that returns a result based on inputs provided by the devices. The inputs
and the result depend on the end devices. For our IQubes the inputs are always the current status
of the cubes in terms of relative position to other cubes and absolute position to the ground, and
the output is a new state that is locally decoded into actions (play sound, display image etc.), but
for other smart devices inputs vary from accelerometer data to camera data and the outputs from
simple Boolean values to complex graphs.
The following figure illustrates the dataflow between end devices and such a workflow
instance.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 54
Figure 21: Dataflow between endpoint devices and Workflow Engine
These workflows do not implement any complex extensions like Tracking or Persistence
since they are stateless and as such will be the ones that can be edited with the workflow editor.
The other workflow type is a state machine workflow. This is more complex, consisting of a
large number of states and transitions between them and as such, the possibility to create them
will not be offered to third parties. Instances of these workflow types control the flow of the
experiment itself, which is broken down into steps such as Predownload data to device, Start
Experiment or Get Score. Triggers for state transitions are both timed based and from external
inputs such as the teachers. As such, the persistence is useful for these types. Tracking will allow
storing of scores that can later be evaluated by persons. An example of such a workflow can be
seen in Appendix I.
Regardless of type, the XAML will be stored as a string inside an SQL Azure table and
retrieved when needed, with the ActivityXamlServices assembly being used.
Activity Library
Another discrete component is the extendable Activity Library, which extends the built-in
activities with some specialized blocks that encapsulate complex functions. These range from
determining the light quantity in a picture to assigning the state of a Cube.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 55
The Library consists of two assemblies:
SequentialActivityLibrary
SequentialActivityLibrary.Design
The first one implements the functional part of a block while the second one implements the
user interface.
The simplest one is the one that controls assignment of a Cube State.
Listing 6: Assign value to Cube Activity
namespace SequentialActivityLibrary { [ToolboxBitmap(typeof(AssignCubeValueActivity), " AssignCubeValueActivity.png")] public sealed class AssignCubeValueActivity : NativeActivity<WorkflowCube[]> { [RequiredArgument] public CubeDevices.Action AssignAction { get; set; } [RequiredArgument] public int CubeNumber { get; set; } [RequiredArgument] public InOutArgument<WorkflowCube[]> DeviceInternals { get; set; } protected override void Execute(NativeActivityContext context) { WorkflowCube[] cubes; cubes = context.GetValue(this.DeviceInternals); cubes[CubeNumber - 1].CubeAction = AssignAction; } } }
The Toolbox bitmap attribute is only used to control which icon is displayed in the Toolbar
of the designer or in the designer itself. The activity extends the NativeActivity class and
overrides only one method, the execute one. It also list the needed arguments, the
RequiredArgument Attribute ensures that the designer will display and error message if nothing
is filled in and will not let anyone continue.
The context object is used to retrieve the Cube arguments which are bound to that particular
activity instance.
The second component is the user interface for the designer; it consists of a XAML that
describes a form. The data is dynamically bound to the values in the form, by means of the
Binding Path attribute, thus the values are automatically recorded on save, and passed to the
CubeActivity object at runtime.
Listing 7: Binding in the user interface XAML
<TextBox Text ="{Binding Path=ModelItem.CubeNumber, Mode=TwoWay}"/>
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 56
The user interface supports two modes, an extended mode and a collapsed mode, bythe use
of a DataTemplate element
Listing 8: Collapsed or expanded DataTemplate
<DataTemplate x:Key="Collapsed"> </DataTemplate>
<DataTemplate x:Key="Expanded"> </DataTemplate>
Rehosted designer
The rehosted designer provides access to an interface similar to that of the IDE. The
Workflow Editor provides a configurable environment to develop experiments, allowing full
control over which activities to display or hide, and which to highlight. Our implementation uses
the Windows Presentation Foundation XAML Browser Application technology. This means that
the whole designer will be available from within the Web Browser. Installation is done by use of
the ClickOnce deployment system, which means that the user will be presented with the choice
to proceed with the software installation or not. The user interface is reminiscent of the designer
in Visual Studio.
Figure 22: Workflow Editor Main Window
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 57
That is because at the core the same object is used, however the rehosted one has to be
configured, since only the view model is offered. The Workflow designer class provides the
basic model for viewing and interacting with the workflow. Adding it to the user interface is
simple; however this only solves part of the problem.
Listing 8: Adding the WorkflowDesigner instance
private void AddDesigner() { this.workflowDesigner = new WorkflowDesigner(); this.workflowDesigner.Load(new Sequence());
var view = workflowDesigner.View; Grid.SetColumn(view, 1); Grid.SetRow(view, 1);
workflowDesigner.ModelChanged += ShowWorkflowXAML; grid1.Children.Add(this.workflowDesigner.View);
}
In order to accommodate the various types that a workflow can have an
AssemblyContextControlItem has to be provided. This is illustrated in the following snippet.
Since adding all Assemblies as reference is not plausible, the selection has to be made carefully
Listing 9: Adding the referenced assemblies
AssemblyContextControlItem acci = new AssemblyContextControlItem(); acci.ReferencedAssemblyNames = new List<AssemblyName>(); acci.ReferencedAssemblyNames.Add(new AssemblyName("LiveXWCFService")); acci.ReferencedAssemblyNames.Add(new AssemblyName(typeof(System.Object ).Assembly.FullName)); this.workflowDesigner.Context.Items.SetValue(acci);
In addition, toolbar items have to be explicitly added to the toolbar, if filtering of provided
activities is desired.
Listing 10: Toolbar creation
private static ToolboxControl CreateToolbox() { var toolboxControl = new ToolboxControl(); toolboxControl.Categories.Add( new ToolboxCategory("Control Flow") { new ToolboxItemWrapper(typeof(DoWhile)), new ToolboxItemWrapper(typeof(ForEach<>)), }); return toolboxControl; }
When creating an experiment, depending on the type chosen, the creator must specify the
information for the devices (eg, in case of smartphone descriptions or pictures) or for cubes
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 58
define states and for each one specify which pictures to display, whether to flash the LEDs,
vibrate, play a sound, etc. These information is stored in the database on the server.
To specify data for an Experiment (such as Description or Images to use) a separate window
is used. There are different pages, and consequently different XAML files for that, depending on
device type.
For the phone, the page is displayed in the next image (Figure x-7) :
Figure 23: Phone Interface
Providing simple and a clean design,it was developed to be intuitive. The listbox on the
right allows assets for the experiments to be added (mainly images for now, further development
will include other types). The limitation comes from the Phone environment and the decision to
provide a unified experience to any supported devices, as both the server and the applications
could handle such data without issues.
The ListBox has an ObservableCollection of objects attached to it, as such the update of the
UI with new files is left to the framework, the only thing needed is to specifically bind it to the
ListBox, which is done in the constructor
Listing 11: Binding an ObservableCollection to the View publicPhoneInterface(PhoneExperimentData data){ …
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 59
ObservableAssetList = new ObservableCollection<Asset>(); AssetList.ItemsSource = ObservableAssetList; AssetList.DataContext = this;
… }
However, there is an issue here, as an ObservableCollection is not serializabile so a List has
to be used. The ObservableCollection has a constructor that can create a Collection from a List
ObservableCollection<T>(List<T>)and aToListMethod extracts a List from
theObservableCollection. This then can be serialized as a member of the ExperimentDataclass
which will then be send to the cloud.
The interface to define Cube States for an experiment is shown in the next figure:
Figure 24: Defining Cube States
Each Cube State lists the possible actions of a Cube in that state. These are displaying an
image on each side, flashing the LEDs, Vibrating or Playing a sound. Again, an observable
collection is used, automatically bound to the ListBox.
An interesting aspect here is the display of the Image inside of the buttons. To do this, the
System.Drawing.Image object has to be converted to a BitmapImage. This is done with the help
of the written method that is displayed in the next Listing. The method saves the image to a
memory stream, in the bitmap format, rewinds the MemoryStream to the first position. That
memory stream is used a stream source for the BitmapImage. The BeginInitand EndInit methods
are important, as without them the stream is not read. This is required because the Image control
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 60
for the button is of a System.Windows.Controls.Image type, and there is no direct or implicit
conversion between those two formats.
Listing 12: Convering an Image to a BitmapImage privateBitmapImageConvertToBitmap(System.Drawing.Imageimg)
{ MemoryStreamms = new MemoryStream(); img.Save(ms, ImageFormat.Bmp); ms.Position = 0; BitmapImage bmp = new BitmapImage(); bmp.BeginInit(); bmp.StreamSource = ms; bmp.EndInit(); return bmp; }
The ExperimentData classes have the following design:
Figure 25: ExperimentData Classes
There are two classes defined for each devices, one holds the various assets or states (such
as pictures for the phone, or the images, LED colors, Audio file etc), and the other holds an
collection of the objects of the first kind, along with other information such as the Description in
case of the Smartphone experiments. These classes implement the IExperimentData interface to
give them a unified architecture, and are the objects which are sent to the cloud to be stored in
the SQL database, or blob storage, along with the XAML.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 61
Implementing tracking
There is no default SQL tracking participant for desktop applications, but there is support
for server instances (which will be using). Enabling a default tracking profile is as trivial a
checking a box in the IIS Manager, as shown in figure x. However this quickly proved to be
inefficient, tracking some trivial information that was not required for business data, while
omitting other, so writing a custom participant was needed. This means we needed to create the
whole store schema with tables.
Figure 26: Enabling built-in tracking
The Tracking related tables inside the database have the following format: A table holds
information regarding the workflow instance, such as its unique ID, the type and various
information such as country of the requester, class etc., so information that is statistically
relevant for teachers or parents, but not something that can be extracted from workflow tracking.
Another two tables hold the tracking events, one records the instance related entries, such as
state changes and one for each underlying activity. The first table is filled explicitly when
invoking the workflow and the other two have records added by automatic invocation of the
TrackingParticipant by the infrastructure.
The relevant information, such as pass/fail or score is computed based on the end state of
the experiment, the duration of one state related to its difficulty level and a par time, number of
tries in a state until moving on to the next and so on.
The format of the tracking related tables can be visualized in the next figure.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 62
Figure 27: SQL Database
Our SQL Tracking Participant extends the TrackingParticipant class and overrides the Track
method, storing the desired events. The Track method will be automatically called by the
infrastructure, so the only thing needed is to register the participant as an extension to the
workflow instance. The Workflow infrastructure will take care of the automatic invocation when
the event occurs. Given the target of the platform, the tracking service is used to store statistical
data such as running time of the experiment or number of runs.
Listing 13: Tracking participant Track function
public class LiveXWfTrackingParticipant : TrackingParticipant { protected override void Track(TrackingRecord record, TimeSpan timeout) { WorkflowInstanceRecord instanceTrackingRecord = record as WorkflowInstanceRecord; if (instanceTrackingRecord != null) { TrackInstance t = new TrackInstance(); t.WorkflowID = instanceTrackingRecord.InstanceId; t.Status = instanceTrackingRecord.State; t.EventDate = DateTime.UtcNow; LiveXWFDataContext dc = new LiveXWFDataContext(_connectionString); dc.TrackInstances.InsertOnSubmit(t); dc.SubmitChanges(); } ActivityStateRecord activityStateRecord = record as ActivityStateRecord;
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 63
if (activityStateRecord != null) { TrackActivity t = new TrackActivity(); t.ActivityName = activityStateRecord.Activity.Name; t.WorkflowID = activityStateRecord.InstanceId; t.Status = activityStateRecord.State; t.EventDate = DateTime.UtcNow; LiveXWFDataContext dc = new LiveXWFDataContext(_connectionString); dc.TrackActivities.InsertOnSubmit(t); dc.SubmitChanges(); } }
}
Going into detail on the code: There are two types of tracking Records, one tracks the
progress of the workflow instance as a whole and one tracks each individual sub activity for
which it is enabled. The records are explicitly inserted into the database with the InsertOnSubmit
and SubmitChanges calls.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 64
Implementing the Windows Phone 7 Client
The windows phone 7 client consists of an application that has a main window filled with
Live Tiles and a secondary page for the experiment as a panorama page which is dynamically
filled with contents from the database. The database fetches the information from the cloud.
Database
The database used, is a local SQL compact database that is stored in the isolated storage. It
consists of two tables that are normalized, one holds only minor information about the
experiment, and the other holds the contents of each experiment, as can be observed in the
following figure.
The second table doesn‘t always have the experiment information to limit resource usage,
instead depending on the last run and the currently deployed experiments, the data may be
deleted to save space and redownloaded once it‘s required, leaving other data to get deleted. The
flag Deployed in the Experiments table specifies whether the data is present or not. The main
assets (pictures and so on), are stored discretely inside the isolated storage as jpegs, to maintain
database performance, since they would not be structured.
Figure 28: Experiment database
The data models generated directly from C#. The tables implement the
INotifiedPropertyChanged and INotifiedPropertyChanging interfaces which allow optimized
access when the database is committed.
The algorithm to replace the data is as follows. If the size of the deployed items added with
the size of the experiment to be downloaded is larger than the allocated size, lookup the
experiments that are deployed and not scheduled, sort them by their last run date ascending and
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 65
delete as many as needed so that the newly downloaded one will keep the total size in limits. The
implementation is shown in the next listing.
Listing 14: Cleaning the database of superfluous experiments varexp = from a in experiments wherea.Deployed&& !a.Deploying&& (a.ScheduledDate == null) orderbya.LastRun ascending select a; if(exp != null && (exp.Sum(item =>item.Size) > EXPERIMENT_SIZES)) { foreach(Experiment a in exp) { if (sum <= result.Size) { sum += a.Size; DeleteExperimentData(a.ExperimentId); a.Deployed = false; } else break; }
User interface
The main page has the newly incorporated Live Titles (Figure X), which are essentially
animated buttons with specific functionality (for example, they can be triggered to show
notifications). The application creates these buttons dynamically based on the information
retrieved from the cloud and stored in the database. The procedure is as follows.
In the constructor the new experiment indexed information that is pushed to the device is
added to the database and based on that database the live tiles are created. Navigation is done
with the use of the Navigation service that uses a URI mechanism with which the selected
experiment can be transmitted. This allows the experiment page to be generic in that regard and
allows it to be generic. The Experiment ID is transmitted as a parameter, and the constructor of
the experiment page fetches the experiment data from the database.
Listing 15: Navigating to the experiment page
NavigationService.Navigate(new Uri( "/ExperimentPage.xaml?experimentID=" + experiment.Name + "?experimentInstance=" + ((result.ScheduledInstance == null) ? "" :result.ScheduledInstance), UriKind.Relative));
The experiment page is a Panorama page consisting of 4 panorama items, that are statically
defined in the XAML file. The text and images is returned from the database and for the
experiment and result page a stack panel is implemented that can be filled with controls from
within IronRuby.Since the experiments require different controls depending on types, we ended
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 66
up having to implement many C# helper methods that call methods from written in Ruby through
the InvokeMembermember.
Sensors
To access sensors I implemented an Interface called ISensors, so that the sensor related
objects can be accessed without any need for Reflection from Ruby. The Interface defines two
methods,StartSensorandStopSensor with which the recording of data by the sensor can be turned
on and off. The Sensors have a polling time, which means that the data retrieved from the sensor
is done at regular intervals using the event handler (egSensorValueChanged).
Listing 16: ISensors Interface public interface ISensor{ voidStartSensor(intpollingtime); voidStopSensor(); }
All the existing, and new classes that gather information from sensors will implement that
interface:
Figure 29: ISensor Interface
Data handling is left to the IronRuby method, which is called from the event handler
method.
Cloud
Cloud connectivity is easily achieved through Windows Communication Foundation. It
allows remote methods to easily be called with serializabile types as a parameters and such types
to be returned. However, since, as stated before, Windows Phone works in an asynchronous way,
a method which registers to the event handler has to be written and subscribed too:
client.GetWP7ResultCompleted += new EventHandler<ServiceClient.GetWP7ResultCompletedEventArgs>(client_getResult);
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 67
private void client_getResult(object sender, ServiceClient.GetWP7ResultCompletedEventArgs e) { //e.Result holds the returned value }
This also means that a mechanism to prevent multiple requests simultaneously had to be
sent has to be implemented. Processing of the data is left to IronRuby methods.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 68
Conclusions
With Windows Workflow Foundation, Microsoft provided developers with the building
block to design and deploy unified, long-running, scalable and maintainable applications.
Workflows provide a proven paradigm that is both tested and innovative, which allows even
non-IT person to quickly learn to develop rich and safe applications. The inclusion of the
workflow designer proves the step in that direction, not restricting development to machines with
Visual Studio installed.
The decision to include workflows as business logic in our platform has proven to be
good one, due to the high level of integration, ease of deployment and further development.
Windows Phone 7 is an operating system that reached maturity, however the decision not
to include any dynamic scripting language can hinder development in certain cases.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 69
IQube: a cube for learning – embedded educational device
Abstract
This project represents a relatively new concept to be used mainly in primary education to
improve the overall quality of education: a learning cube, tangible, interactive and playing
platform for children. Our cube is very different from other educational cubes because it has a
colored display on each of its sides, making it very customizable. Also, a cube is not to be used
as a standalone learning tool. It is to be associated with other cubes in games like word matching,
picture association or any other games that require multiple matching. Another aspect that makes
the cubes attractive is the fact that they have accelerometer sensors creating new possibilities for
user interaction, very attractive, especially to children.
IQubes are electronic smart cubes with an LCD screen on each of its side, with many
sensors and capable of communicating with other IQubes and with the cloud. They are mainly
used as a hands-on user interface for an experiment. They can be combined in many different
ways to perform a large variety of experiments
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 70
Introduction
It is known that the development of knowledge has a direct bearing on a person‘s
individuality. Every person should be able to attain education. It is a fundamental human right
that all people must have, but this is nowhere near accomplished for some people. Some of the
barriers that prevent this may include its cost, making it unaffordable for some people, or its
decreasing quality. Considering that recent studies from UNESCO present some alarming facts
related to literacy [BAME02], we present a new concept to be used, mainly, in primary education
to address this problem: a learning cube, tangible, interactive and playing platform for children.
Our solution to address this problem is to try and make education more accessible and fun
for children from ages 2 to 10. Children prefer to challenge the world: explore, talk aloud, move
around and manipulate different objects instead of interacting with the virtual world shaped by
computers [DAIK01].
The design and development of an IQube relies on two main considerations which make our
proposed solution stand out when compared to other conventional learning methods: in the real
world children learn better by playing [PJHC03] and building representational mappings is
facilitated by performing physical activities [WRGE06]. The IQube platform is based on Mind
Lab‗s learning process [WMLE07] which begins with children engaging in game-playing
activities and ultimately empowers them with skills and knowledge relevant to real-life
situations.
The main advantage of our system is the flexibility of applying it into many areas. Besides
its main functionality, as a learning tool, our cube can be used for browsing many kinds of
contents such as Blogs, Web news, Movies, Music Videos and so on. In this way, the IQube may
be used by various users. Moreover, progression through play reflects the observation and
assessment of children‗s knowledge, skills and attitudes in order to provide developmentally
appropriate experiences. Children will go to school already as skilled learners. Through
observations on how every child plays with the IQube and solves different puzzles, we gain
valuable insights into how each one learns best. This kind of information may help the teachers
and the parents learn more about the children‗s needs and can better understand that child‗s
abilities, interests and potential.
We have conceived the IQube, to be a low cost and flexible hardware/software edutainment
platform. Its flexible structure allows it to be reutilized for many years, decreasing the overall
cost of implementing the solution.
One crucial aspect is the user interface; our team concluded after research that the children
must interact naturally with the educational device, without any forehand training. So we
excluded classical PC interfaces such as mouse, keyboard, etc. and came up with the idea of
creating cube shaped devices. There is no need for instructions how to use it. There are no
special rules for how to play with it. It is simple, colored, plays different sounds and vibrates.
We designed the IQube in order to cover very different fields: literacy, mathematics,
chemistry, logical games and even music. For the number of games that can be developed for the
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 71
IQube, imagination is the limit. For creation of a wide portfolio of games, experiments and
educational applications to be uploaded on an IQube website in order to be accessed by the
IQube educational community worldwide, a wide volunteering effort could be triggered, with
people crowd-sourcing new valuable community-rated materials, generating a real IQube
revolution.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 72
Device description
Mechanical aspects
In order to visualize the cube that we will construct, it was
necessary to make a 3D model before the actual construction of
the device. Using the 3D model we tried to arrange the
component s inside so that we have optimum efficiency
regarding placement and free space distribution. It was also very
useful in helping us finding the correct battery pack so that it will
fit inside the case.
The final size for the cube is 59x59x59 mm, on each of its
size there is a pocket 29x29mm where the 1.6‘ LCD is mounted.
The plastic of the case is white with a 40% transparency index. This
is very useful because the light from the RGB LEDs inside the cube
is dispersed and makes the cube uniformly light up.
Electronic aspects
The cube is powered by a 3.7 Volt Lithium-ion battery which
is capable of keeping it powered up for about 3 hours. The voltage of the whole circuitry is 3.3 V
and is provided by a very low drop voltage regulator (LF33CDT).
In order to detect other cubes we have decided to use hall sensors. These sensors are placed
inside a ring magnet at the back of each LCD. The decision to choose this placement has been
taken after careful experimentation with several types of magnets.
Fig. 30: Inside the chasis
Fig. 31: Chasis
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 73
Architecture
The IQube platform is based on a low-power microcontroller. It has wireless
communication capabilities and several built-in sensors and actuators (e.g., an accelerometer and
a vibrating motor).
LCD 1 LCD 2 LCD 6
Speaker Haptic actuator
OutputRGB status LED
Control
Xbee
Input
Accelerometer Hall sensors
RS 232Xbee
C# Application
8 bit Microcontroller
Coordinator
Storage device(SD/MMC card)
SDRAM Memory
Power
Battery
Charging circuit(may be wireless)
IQube
Fig. 32: Component block diagram
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 74
A single IQube has a 1.6' LCD on each side of the cube which is its defining feature, a
speaker for emitting sounds, a small vibrating motor and a RGB status led for indicating the
status of that cube and position (relative to other cubes).
The user interacts with the cube by means of an accelerometer sensor. Each tilt and rotation
of the IQube corresponds to a certain change in its state. Also, the position relative to other cubes
is constantly monitored using internal hall-effect sensors. The cube‘s state and position is sent to
an embedded central node (C.E.N.) via wireless communication. The central node then processes
the information received and, depending on the displayed images and the cube‗s position, it will
send other images, change the color of the cube, announce a game completion or failure,
depending on the running app.
Transmission and reception of information is done using a wireless connection between the
IQube and the C.E.N. One way of accomplishing this is by using an XBee [WXBE08] wireless
communication module. At the heart of the device lies a microcontroller which controls all above
mentioned components. In addition, a storage component is added to store the received
information and media (picture, sounds or animations) from the C.E.N. The power supply
consists of a rechargeable battery which could be recharged wireless using inductive coupling.
As mentioned above, all the cubes communicate to one another as an ad-hoc network
communication, as well as with the C.E.N. The next figure (fig.33) shows one possibility of how
this could be done.
Inductive charging pad
C.E.NC.E.N
touchscreentouchscreen
2 two
Fig. 33: Ad-hoc network
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 75
Embedded Device Hardware Modules
Internal
1) Microcontroller
The central element of the IQube is the microcontroller. We used an 8-bit AVR XMega
microcontroller for this task. We chose this microcontroller because of its low power, low
voltage operation and a relatively high speed (32 MHz)
We chose this microcontroller mainly because it has a lot of peripherals. In our project we
are using the following peripherals:
The external bus interface (EBI) – used for communication with an external SDRAM
memory,
I2C – used for the accelerometer
Serial peripheral interface (SPI) – used for reading/writing data to/from the SD card and
for displaying images on the LCDs.
The Universal Synchronous and Asynchronous serial Receiver and Transmitter (USART) -
used for communication with the wireless XBee module.
Analog to Digital Converter (ADC) – used to read the output from the Hall-Effect sensors
Digital to Analog Converter (DAC) – used to convert digital audio and send it to the
speaker
GPIO ports used to control the RGB LEDs, and the haptic actuator.
JTAG interface – for programming and debugging
The speed at which we work with the microcontroller is 29.491200 Mhz. This frequency is
obtained using an external crystal oscillator at 14.745600 MHz and multiplied by 2x using the
PLL the microcontroller provides.
A block diagram of the internal components of the microcontroller is displayed in the
next figure:
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 76
Fig. 34: ATXMega MicroController Diagram
2) SDRAM Memory
To connect extra memory to the
controller we used the External bus
interface of the microcontroller.
The XMEGA™ External Bus
Interface (EBI) is a highly flexible
module for interfacing external memories
and memory addressable peripherals such
as LCD Controllers and advanced
communication controllers. The EBI Fig. 35: Connecting Memory with EBI
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 77
module has four separate Chip Select blocks with individual address ranges and waits state
control. Additional Chip Select lines can be decoded externally. Flexible settings for address
multiplexing and external latches,individual settings for the four Chip Select blocks and
transparent support for SDRAM with automatic refresh make this module the perfect match for
all applications using external memories and addressable peripherals. [DSDRA04]
The EBI is connected with the external memory as shown in figure x.
The memory used is Micron® 64Mb SDRAM (MT48LC16M4A2 [sdram
datasheet] - fig. x), high-speed CMOS, dynamic random-access memory
containing 67,108,864 bits. It is internally configured as a quad-bank DRAM
with a synchronous interface (all signals are registered on the positive edge of
the clock signal, CLK).
3) NAND flash memory (SD/MMC card)
The microcontroller communicates with
the SD Card using the SPI Interface. The SPI
Interface is configured to work at the highest
possible speed supported by the
microcontroller with the current XTAL
oscillator (14.745600 MHz).
Input
1) Hall sensors.
The hall sensors are used for the cube to detect other cubes.
Each cube has one ring magnet and one hall sensor behind the
LCD display (fig 40).
The hall sensors are read by the microcontroller using an ADC
port. The output of a sensor is an analogicalvoltage that is
directly related to the intensity of the magnetic field. When any 2
sides from any 2 cubes come in contact the hall sensors will
8 bit Microcontroller
SPI Interface
Fig. 36: SDRAM Module
Fig. 37: Interconnecting SD
Fig. 39: SD Connection Fig. 38: SD Layout
Fig. 40: Hall sensor
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 78
detect the field coming from its surrounding magnet as well as from the ring magnet in which it
came in contact with. (fig 42)
2) Accelerometer
Another input for our device is the
accelerometer. We used the MMA8452Q
accelerometer produced by freescale. Some of its
features are as follows: low-power, three-axis, 12
bits resolution, embedded functions with flexible
user programmable options, configurable to two
interrupt pins. Embedded interrupt functions allow
for overall power savings relieving the host
processor from continuously polling data.
[ACCD01]
Output
1) Speaker
In order to play sounds we equipped our
device with a low-cost, high performance
speaker. It has 1.4 W and 8ohm impedance.
The microcontroller generates the sound wave using its internal DAC and it sends it to a class D
audio amplifier (TPA2005D1DGN).
8 bit Microcontroller
Hall sensor
Hall sensor
Hall sensor
Hall sensor
Hall sensor
Hall sensor
ADC
ADC
ADC
ADC
ADC
ADC
Hall sensorMagnet
8 bit Microcontroller
I2C (TWI) Interface Acceleromter
Fig. 41: Connecting Hall Sensors
Fig. 42: Magnet and Hall sensor
Fig. 43: Acclerometer
Fig. 44: Connecting the Speaker
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 79
2) RGB LEDs
To be able to change the case color of the cube we used RGB LEDs. Using these kind of
LEDs all colors can be reproduced using the PWM Output of the microcontroller
8 bit Microcontroller
GPIO (PWM Output)
Transistors
8 X RGB LEDs
3) Haptic actuator
To further provide feedback to the user we used an haptic actuator, connected as shown
below:
8 bit Microcontroller
GPIO (PWM Output)
Transistor
4) LCD displays
The interface used to send images to the display is the SPI. All the
displays are connectedto the same bus and a Chip Select pin is used to select
only a certain display to show the image on.
8 bit Microcontroller
LCD Display
SPI BUS
LCD Display
LCD Display
LCD Display
LCD Display
LCD Display
The LCD Display used is 1.6‘, has 65k colors codified using the RGB 5-6-6 standard.
Fig. 45: Connecting the RGB LEDs
Fig. 46: Connecting the hapting actuator
Fig. 47: Connecting the LCD Displays
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 80
Control
The cube communicates with the coordinator using an xbee wireless communication
module. Each cube and the coordinator (laptop computer) have their own xbee module. They are
all configured to work in broadcast mode, and the selection of the packets is made using our
defined protocol at the application layer. The module is connected to the microcontroller using
the USART interface (fig 48) and with the computer using a serial interface or, in our case, a
USB->SERIAL converter
`
Xbee
8 bit Microcontroller
USART Interface
Fig. 48: ZIGBee interface (XBee)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 81
Embedded Device Application
In this chapter we describe a few key aspects of the embedded device application.
SD Card and FAT system
The images, videos, and sounds are all stored on an external SD card. The protocol used for
communication with it using a microcontroller with an SPI interface is described below:
[MMCF06]
SPI Mode
The SPI mode is an alternative operating mode that defined to use the MMC/SDCs without
native host interface. The communication protocol for the SPI mode is simple compared to its
native operating mode. The MMC/SDC can be attached to the most microcontrollers via a
generic SPI interface or GPIO ports. Therefore the SPI mode is suitable for low cost embedded
applications. Especialy, there is no reason to attempt to use native mode with a cheap
microcontroller that has no native host interface. For the SDC, the SPI mode 0 is defined for its
SPI mode. For the MMC, it is not the SPI timing, both latch and shift actions are defined with
rising edge of the SCLK, but it seems work in SPI mode 0 in SPI mode. Thus the SPI Mode 0
(CPHA=0, CPOL=0) is the proper setting for MMC/SDC interface, but SPI mode 3 also works
as well in most case.
Command and Response
In SPI mode, the data direction on the signal line is fixed and the data is transferred in byte
oriented serial communication. The command frame from host to card is a fixed length (six
bytes) packet that shown below. When a command frame is transmitted to the card, a response to
the command (R1, R2 or R3) is sent back from the card. Because the data transfer is driven by
serial clock generated by host, the host must continue to read bytes until receive any valid
response. The DI signal must be kept high during read transfer (send a 0xFF and get the received
data). The response is sent back within command response time (NCR), 0 to 8 bytes for SDC, 1
to 8 bytes for MMC. The CS signal must set low prior to send a command frame and held low
during the transaction (command, response and data transfer if exist). The CRC feature is
optional in SPI mode but the CRC field is always required to compose a command frame.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 82
SPI Command Set
Each command is expressed in abbreviation like GO_IDLE_STATE or CMD<n>, <n> is
the number of the command index and the value can be 0 to 63. Following table describes only
commands that to be usually used for generic read/write and card initialization. For details on all
commands, please refer to spec sheets from MMCA and SDCA.
Command Index Argument Response Data Abbreviation Description
CMD0 None(0) R1 No GO_IDLE_STATE Software reset.
CMD1 None(0) R1 No SEND_OP_COND Initiate initialization process.
ACMD41(*1) *2 R1 No APP_SEND_OP_COND For only SDC. Initiate initialization process.
CMD8 *3 R7 No SEND_IF_COND For only SDC V2. Check voltage range.
CMD9 None(0) R1 Yes SEND_CSD Read CSD register.
CMD10 None(0) R1 Yes SEND_CID Read CID register.
CMD12 None(0) R1b No STOP_TRANSMISSION Stop to read data.
CMD16 Block length[31:0] R1 No SET_BLOCKLEN Change R/W block size.
CMD17 Address[31:0] R1 Yes READ_SINGLE_BLOCK Read a block.
CMD18 Address[31:0] R1 Yes READ_MULTIPLE_BLOCK Read multiple blocks.
CMD23 Number of blocks[15:0] R1 No SET_BLOCK_COUNT
For only MMC. Define number of blocks to transfer with next multi-block read/write command.
ACMD23(*1) Number of blocks[22:0] R1 No SET_WR_BLOCK_ERASE_COUNT
For only SDC. Define number of blocks to pre-erase with next multi-block write command.
CMD24 Address[31:0] R1 Yes WRITE_BLOCK Write a block.
CMD25 Address[31:0] R1 Yes WRITE_MULTIPLE_BLOCK Write multiple blocks.
CMD55(*1) None(0) R1 No APP_CMD Leading command of ACMD<n> command.
CMD58 None(0) R3 No READ_OCR Read OCR.
*1:ACMD<n> means a command sequense of CMD55-CMD<n>.
*2: Rsv(0)[31], HCS[30], Rsv(0)[29:0]
*3: Rsv(0)[31:12], Supply Voltage(1)[11:8], Check Pattern(0xAA)[7:0]
Fig. 49: SPI Command
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 83
SPI Response
There are three command response
formats, R1, R2 and R3, depends on the
command index. A byte of response R1 is
returned for most commands. The bit field of
R1 response is shown in right image, the value
0x00 means successful. When any error
occurred, corresponding status bit in the
response will be set. The R3 response (R1 and trailing 32-bit OCR) is for only CMD58.
Some commands take a time longer than NCR and it responds R1b. It is an R1 response
followed by busy flag (DO is held low as long as internal process is in progress). The host
controller should wait for end of the process until DO goes high (0xFF is received).
Initialization Procedure for SPI Mode
After power on reset, MMC/SDC enters its native operating mode. To put it SPI mode,
following procedure must be performed like this flow.
Power ON (Insertion)
After supply voltage reached 2.2 volts, wait for a millisecond at least. Set SPI clock rate
between 100kHz and 400kHz. Set DI and CS high and apply 74 or more clock pulses to SCLK.
The card will enter its native operating mode and go ready to accept native commands.
Software Reset
Send a CMD0 with CS low to reset the card. The card samples CS signal on a CMD0 is
received successfully. If the CS signal is low, the card enters SPI mode and responds R1 with In
Idle State bit (0x01). Since the CMD0 must be sent as a native command, the CRC field must
have a valid value. When once the card enters SPI mode, the CRC feature is disabled and the
CRC is not checked by the card so that command transmission routine can be written with the
hardcoded CRC value that valid for only CMD0 and CMD8 with the argument of zero. The CRC
feature can also be switched with CMD59.
Initialization
In idle state, the card accepts only CMD0, CMD1, ACMD41,CMD58 and CMD59. Any
other commands will be rejected. In this time, read OCR register and check working voltage
range of the card. In case of the system supply voltage is out of working voltage range, the card
must be rejected. Note that all cards work at supply voltage range of 2.7 to 3.6 volts at least. The
Fig. 50 SPI Response
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 84
card initiates initialization when a CMD1 is received. To detect end of the initialization process,
the host controller must send CMD1 and check the response until end of the initialization. When
the card is initialized successfully, In Idle State bit in the R1 response is cleared (R1 resp
changes 0x01 to 0x00). The initialization process can take hundreds of milliseconds (large cards
tend to longer), so that this is a consideration to determine the time out value. After the In Idle
State bit cleared, generic read/write commands will able to be accepted.
Because ACMD41 instead of CMD1 is recommended for SDC, trying ACMD41 first and
retry with CMD1 if rejected, is ideal to support both type of the cards.
The SPI clock rate should be changed to fast as possible to maximize the read/write
performance. The TRAN_SPEED field in the CSD register indicates the maximum clock rate of
the card. The maximum clock rate is 20MHz for MMC, 25MHz for SDC in most case. Note that
the clock rate will able to be fixed to 20/25MHz in SPI mode because there is no open-drain
condition that restricts the clock rate.
The initial read/write block length can be set 1024 on 2GB cards, so that the block size
should be re-initialized to 512 with CMD16 to work with FAT file system. unsignedcharmmc_init() { unsignedintTimeout=0; /* Init SS pin as output with wired AND and pull-up. */ PORTD.DIRSET=PIN4_bm; PORTD.PIN4CTRL=PORT_OPC_WIREDANDPULL_gc; /* Initialize SPI master on port C. */ spi_master_init(MMC_SPI); spi_master_setup_device(MMC_SPI,&MMC_SPI_DEVICE,SPI_MODE_0,SPI_EXAMPLE_BAUDRATE,0) spi_enable(MMC_SPI); MMC_Enable(); //Initialize MMC/SD Card in SPI mode for(unsignedcharb=0;b<0x0f;b++)//Sende min 74+ Clocks to SD card { mmc_write_byte(0xff); } //Sendet Command CMD0 to SD card unsignedcharCMD[]={0x40,0x00,0x00,0x00,0x00,0x95}; while(mmc_write_command(CMD)!=1) { if(Timeout++>200) { MMC_Disable(); return(1);// (Return Code1) } } //Send CommandCMD1 to SD card Timeout=0;
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 85
CMD[0]=0x41;//Command 1 CMD[5]=0xFF; while(mmc_write_command(CMD)!=0) { if(Timeout++>400) { MMC_Disable(); return(2);(Return Code2) } } MMC_Disable(); return(0); }
How to support SDC Ver2 and high capacity cards
After the card enters idle state with a CMD0, send a CMD8 with argument of 0x000001AA
and correct CRC prior to initialization process. When the CMD8 is rejected with an illegal
command error (0x05), the card is SDC V1 or MMC. When the CMD8 is accepted, R7 response
(R1(0x01) and trailing 32 bit data) will be returned. The lower 12 bits in the return value 0x1AA
means that the card is SDC V2 and it can work at voltage range of 2.7 to 3.6 volts. If not the
case, the card must be rejected. And then initiate initialization with ACMD41 with HCS (bit 30).
After the initialization completed, read OCR register and check CCS (bit 30). When it is set,
subsequent data read/write operations that described below are commanded in block address
instead of byte address. The block size is always fixed to 512 bytes.
Data Transfer - Data Packet and Data Response
In a transaction with data transfer,
one or more data blocks will be
sent/received after command response.
The data block is transferred as a data
packet that consists of Token, Data
Block and CRC. The format of the data
packet is show in right image and there
are three data tokens. As for Stop Tran
token that means end of multiple block
write, it is used in single byte without data block and CRC.
Fig. 51: DataPacket Format
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 86
Single Block Read
The argument specifies the location to start
to read in unit of byte or block. The sector
address specified by upper layer must be scaled
properly. When a CMD17 is accepted, a read
operation is initiated and the read data block will be sent to the
host. After a valid data token is detected, the host controller
receives following data field and two byte CRC. The CRC bytes must be flushed even if it is not
needed. If any error occurred during the read operation, an error token will be returned instead of
data packet.
//Routine to read the CID register of the MMC / SD card (16Bytes) Voidmmc_read_block(unsignedchar*cmd,unsignedchar*Buffer,unsignedintBytes) { //Sendet Command to SD Card if(mmc_write_command(cmd)!=0) { return; } //Wait for start bytes of the MMC / SD card (FEh / Start Byte) while(mmc_read_byte()!=0xfe){}; //Read the Bolck (normally 512 bytes) from MMC / SD card for(unsignedinta=0;a<Bytes;a++) { *Buffer++=mmc_read_byte(); } //CRC-Byte read mmc_read_byte();//CRC - byte is not evaluated mmc_read_byte();//CRC - byte is not evaluated //set MMC_Chip_Select to high (MMC/SD-Card Inactive) MMC_Disable(); return; }
Multiple Block Read
The Multiple Block Read command reads multiple blocks in sequence from the specified
address. When number of transfer blocks has not been specified before this command, the
transaction will be initiated as an open-ended multiple block read, the read operation will
continue until stopped with a CMD12. The received byte immediately following CMD12 is a
Fig. 52: Single Block Read
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 87
stuff byte, it should be discarded before receives the response of the CMD12.
//############################################################################ //Routine for reading a Block (512Byte) from the MMC/SD-Card unsignedcharmmc_read_sector(unsignedlongaddr,unsignedchar*Buffer) //############################################################################ { //Command 16 to read a block from the MMC / SD - card unsignedcharcmd[]={0x51,0x00,0x00,0x00,0x00,0xFF}; /* The address of the MMC / SD card is in bytes, addr will be converted into bytes of block after these are inserted into the command */ addr=addr<<9;//addr = addr * 512 cmd[1]=((addr&0xFF000000)>>24); cmd[2]=((addr&0x00FF0000)>>16); cmd[3]=((addr&0x0000FF00)>>8); mmc_read_block(cmd,Buffer,512); return(0); }
FAT16 file allocation table - 16bit
Short Overview
Fig. 53: Multiple Block read
Fig. 54: FAT overview
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 88
This is a scale diagram of the start of the disk with a small fat16 partition at the beginning.
this partition is just over 20mb in size, to save space not all the data/subdir/surplus area is shown
- there would be 44,815 blocks or ~=896 lines if this part was shown completely.
The master boot record (mbr) is located in the first data sector on the hard disk. the mbr can
have up to 4 primary partitions, each entry occupies 16 bytes.
voidfat_init(void) {
structBootSec*bootp;// Pointer to Boot Sector Structure unsignedcharBuffer[BlockSize]; mmc_read_sector(MASTER_BOOT_RECORD,Buffer);//Read Master Boot Record if(Buffer[510]==0x55&&Buffer[511]==0xAA) { usart_puts(CURRENT_USART,"\r\nMBR Signatur found!\r\n"); } else { usart_puts(CURRENT_USART,"\r\nMBR Signatur NOT found!\r\n"); while(1); } volume_boot_record_addr=Buffer[VBR_ADDR]+(Buffer[VBR_ADDR+1]<<8); mmc_read_sector(volume_boot_record_addr,Buffer); if(Buffer[510]==0x55&&Buffer[511]==0xAA) { usart_puts(CURRENT_USART,"\r\nVBR Signatur found!\r\n"); } else { usart_puts(CURRENT_USART,"\r\nVBR Signatur not found!\r\n"); volume_boot_record_addr=MASTER_BOOT_RECORD; mmc_read_sector(MASTER_BOOT_RECORD,Buffer);//Read Master Boot Record } bootp=(structBootSec*)Buffer; cluster_size=bootp->BPB_SecPerClus; fat_offset=bootp->BPB_RsvdSecCnt; cluster_offset=((bootp->BPB_BytesPerSec*32)/BlockSize); cluster_offset+=fat_root_dir_addr(Buffer); }
SDRAM Memory
Because the application works with large files compared to other embedded application it
was necessary to use external SDRAM memory. For this we have defined a memory structure in
order to work with
structSDRAM_file { unsignedcharfile_name[12]; uint64_tfile_index; uint64_tfile_size; }
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 89
In order to simply all work with the files from the SD Card, we use the 8.3 file format to
read/write data to/from the card and to/from the ram memory.
Next, we present an example function used to copy a file from the SD Card and store it in
the RAM memory.
unsignedlongcopy_file_to_ram(unsignedchar*nume) { uint64_tcluster=0; uint64_tsize=0,count; unsignedchardir_attr=0; unsignedcharbuffer[512]; unsignedcharreturnValueSearch; uint64_tnr=0; uint64_taddressValue; uint64_tlocation; if(lastCopiedFile!=0) { location=ram_file[lastCopiedFile-1].file_size+ram_file[lastCopiedFile-1].file_index; } else { location=0; } returnValueSearch=fat_search_file(nume,&cluster,&size,&dir_attr,buffer); if(returnValueSearch==1) //FILE FOUND { for(count=0;count<=size;) { fat_read_file(cluster,buffer,nr); for(uint32_tj=0;j<512;j++) { addressValue=BOARD_EBI_SDRAM_BASE+location+count+j; hugemem_write8(addressValue,buffer[j]); } count+=512; nr++; } strncpy(ram_file[lastCopiedFile].file_name,nume,strlen(nume)); ram_file[lastCopiedFile].file_index=location; ram_file[lastCopiedFile].file_size=size; lastCopiedFile++; } if(returnValueSearch==0) //FILE NOT FOUND { usart_puts(CURRENT_USART,"FILE NOT FOUND"); play_song("error.wav"); usart_puts(CURRENT_USART,strcat("Unable to copy file: ",nume)); } if(returnValueSearch==2) { usart_puts(CURRENT_USART,"ERROR");
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 90
play_song("error.wav"); usart_puts(CURRENT_USART,strcat("Unable to copy file: ",nume)); } returnsize; }
Embedded application for the Hall-effect sensors
The output from the hall-effect sensor is read using the ADC of the microcontroller.
The ADC has 12-bit resolution and is capable of converting up to 2 million samples per
second. The input selection is flexible, and both single-ended and differential measurements can
be done. For differential measurements an optional gain stage is available to increase the
dynamic range. In addition several internal signal inputs are available. The ADC can provide
both signed and unsigned results. [xmega datasheet]
In the next section we describe how this is achieved in our application with some example
functions and data collected from the sensors. uint16_tpin3_ADCValue() //left { uint64_tadc_sum=0; adcch_set_input(&adcch_conf,ADCCH_POS_PIN3,ADCCH_NEG_PIN7,16); adcch_write_configuration(&ADCA,0,&adcch_conf); for(uint8_tj=0;j<32;j++) { adc_start_conversion(&ADCA,ADC_CH0); adc_wait_for_interrupt_flag(&ADCA,ADC_CH0); adc_sum+=adcch_get_unsigned_result(&ADCA,0); } adc_sum>>=5; returnadc_sum; }
For every ADC pin that has a hall-effect sensor connected to it we read 32 values and make
an average for those values in order to increase the accuracy of the measured data.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 91
In this chart we have the output from a single hall-effect sensor. The output of the sensor
varies when 2 cubes are near each other. This variation is processed and a result is given, as
shown in the example below.
if(pin3_ADCValue()<64600) { usart_puts(CURRENT_USART,"connection: left"); }
Digital to analog converter and wave file format
To easy read and play sound files we chose
the .wav file format. [WAVE06]
The WAVE file format is a subset of
Microsoft's RIFF specification for the storage of
multimedia files. A RIFF file starts out with a file
header followed by a sequence of data chunks. A
WAVE file is often just a RIFF file with a single
"WAVE" chunk which consists of two sub-
chunks -- a "fmt" chunk specifying the data
format and a "data" chunk containing the actual
sample data.
63000
63500
64000
64500
65000
65500
66000
1
12
23
34
45
56
67
78
89
10
0
11
1
12
2
13
3
14
4
15
5
16
6
17
7
18
8
19
9
21
0
22
1
23
2
24
3
25
4
26
5
27
6
28
7
Series1
DIGITALINPUT VOLTAGE 1 unit = 50µV
Number of values measured
Fig. 56 Output from Hall sensor
Fig. 57: The Canonical Wave File Format
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 92
Our cube plays files at the sample rate of 22500HZ, 8 bits of data for each sample. To
ensure optimum playback speed and no load on the CPU, the audio file must first be copied to
the RAM memory.
LCD display embedded application
In order to show an image on an LCD Display as fast as possible we first had to copy that
file to the SDRAM Memory. The next function is an example of how an image is displayed on
an LCD Display
voidshowPictureOnLCDFromRAM(unsignedchar*nume,uint8_tlcdNumber) { uint8_ti; for(i=0;i<MAX_NUMBER_OF_FILES;i++) //search for sound file through RAM memory { if(strncmp(ram_file[i].file_name,nume,12)==0) { if(lcdNumber==0) LCD_activateAll(); else LCD_select(lcdNumber); LCD_WindowSet(1,130,1,130); ioport_set_pin_high(LCD_DC); for(uint16_tj=0;j<33800;j++) //send raw image to LCD { spi_put(LCD_SPI,hugemem_read8(BOARD_EBI_SDRAM_BASE+j+ram_file[i].file_index)); while(!spi_is_tx_ok(LCD_SPI)); } } } }
Communication with the coordinator
The communication with the coordinator is made using wireless xbee modules interfaced
through the USART interface. The next section show an example of how commands are received
and handled by a cube using the protocol that will be described in the next chapter. volatileuint8_t received_byte; volatileuint8_treceived_song_name[12]={'#','#','#','#','#','#','#','#','.','w','a','v'}; volatileuint8_treceived_picture_name[12]={'#','#','#','#','#','#','#','#','.','b','i','n'}; ISR(USARTD0_RXC_vect) //check received bytes
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 93
{ cli(); uint8_treceived_code=usart_getchar(CURRENT_USART); if(received_code==0xa1) { if(usart_getchar(CURRENT_USART)==0xa2) { if(usart_getchar(CURRENT_USART)==0X02) { if(usart_getchar(CURRENT_USART)==(uint8_t)CURRENT_COORDONATOR) { received_byte=usart_getchar(CURRENT_USART); if(received_byte==0xf1) //vibrate { asm("NOP"); MACRO_VIBRATE_ON; _delay_ms(300); MACRO_VIBRATE_OFF; } if(received_byte==0xf2) //case color { setCaseColor(usart_getchar(CURRENT_USART)); } if(received_byte==0xf3) //play song { for(uint8_tindex=0;index<8;index++) { received_song_name[index]=usart_getchar(CURRENT_USART); } play_song(received_song_name); } if(received_byte==0xf4) //show image { uint8_ttempLcdNr=usart_getchar(CURRENT_USART); for(uint8_tindex=0;index<8;index++) { received_picture_name[index]=usart_getchar(CURRENT_USART); } showPictureOnLCDFromRAM(received_picture_name,tempLcdNr); } } } } } sei(); }
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 94
Protocol and Data flow
A communication protocol has been designed in order for the cubes to efficiently
communicate with one another and with the coordinator. In the next section some of the features
of the protocol (packets) are described:
All packets contain 2 bytes used for control. They are chosen in such a way that it could be
very hard for a normal data transfer to contain those 2 bytes and to interfere with the rest of the
packet‘s content. After these control bytes there are 2 more byes that represent the source and the
destination of the sender/receiver. If a device receives a packet and manages to correctly identify
the control bytes, the next byte represents the destination code for that packet. If the packet‘s
destination does not match with the devices‘ code the packet is automatically rejected. Next, we
have the source of the packet. This is particularly useful for the coordinator to know exactly what
cube has sent the packet. The rest of the packet varies and are described in the section below:
IQube to Coordinator
The IQube sends its status to the coordinator. This packet contains all the required
information for the coordinator in order to decide whether a game/experiment solution is correct.
0xa1 0xa2 0xc1 0x02 10100101 picture1
Figure 58: Packet format for request
The connection status bytes represents whether or not a face is connected to another cube.
―1‖ means we have a connection ―0‖ means that the specific side is not connected. The side
encoding is as follows: not used
not used back front right left bottom top
top picture name (8 bytes)
connection status (1 byte)
source (1 byte)
destination (1 byte)
safety code (2 bytes)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 95
Coordinator to IQube
1) Vibrate function (0xf1)
0xa1 0xa2 0x01 0xc1 0xf1
Figure 59: Packet format for Vibrate reply
2) Case color function (0xf2)
0xa1 0xa2 0x01 0xc1 0xf2 0x02
Possible values for color code: NO_COLOR 0x00 RED 0x01 GREEN 0x02 BLUE 0x03 YELLOW 0x04 MAGENTA 0x05 AQUA 0x06 WHITE 0x07
Figure 60: Packet format for flash LED reply
function code (1 byte)
source (1 byte)
destination (1 byte)
safety code (2 bytes)
color code (1 byte)
function code (1 byte)
source (1 byte)
destination (1 byte)
safety code (2 bytes)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 96
3) Play audio function (0xf3)
0xa1 0xa2 0x01 0xc1 0xf3 Song1
Figure 61: Packet format for Play Audio Reply
4) Show image on LCD function (0xf4)
0xa1 0xa2 0x01 0xc1 0xf4 e2 picture1
Figure 62: Packet format for show image on LCD reply
Possible values for LCD number: CUBE_SIDE_ALL 0xe0 CUBE_SIDE_TOP 0xe1 CUBE_SIDE_BOTTOM 0xe2 CUBE_SIDE_LEFT 0xe3 CUBE_SIDE_RIGHT 0xe4 CUBE_SIDE_FRONT 0xe5 CUBE_SIDE_BACK 0xe6
audio file name (8 bytes)
function code (1 byte)
source (1 byte)
destination (1 byte)
safety code (2 bytes)
audio file name (8 bytes)
lcd number (1 byte)
function code (1 byte)
source (1 byte)
destination (1 byte)
safety code (2 bytes)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 97
Coordinator Application
In order for the cubes to communicate with the cloud we have conceived a C# application.
This application is used for acquiring experiments from the cloud,
configure them and run them using the cubes as a tangible user
interfaces. The application connects to the xbee module via a serial
interface.
The application sends and receives data from/to the cubes using the
protocol described in the previous chapter.
Fig. 63: Coordinator Interface
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 98
Similar products and costs
Similar products
After some research, we found many different products in this niche market of education. Some
of them are traditional; others make use of different technologies. In order to identify the major
strengths and drawbacks of our proposal, a competitor analysis contemplating more than 20
competitors was carried out. A summary of such an analysis is presented in this subsection.
We grouped the competitors in 5 categories labeled from A to E:
A) Traditional children books
B) Educative board games
C) Educative software/websites
D) External agents
E) Other educational gadgets e.g. Shiftables
In order to compare these solutions we took into consideration following aspects:
interaction (INT)
interaction between children/team-game
possibility (IBC)
update capability (UC)
cost (COS)
personalization (PER)
Product
I
N
T
I
B
C
UC
C
O
S
P
E
R
T
O
T
A (Traditional
children books) 2 1 1 3 1 8
B (Educative
board games) 2 5 1 2 1 11
C (Educative
software/websites) 3 1 5 5 4 18
D (External
agents) 3 3 2 2 2 12
E (other
educational
gadgets e.g.
Shiftables)
4 3 5 4 4 20
IQube 5 4 5 3 5 22
Fig 64: Some of the competitors
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 99
It is clear that the IQube is not the winner in all the categories. However, overall the IQube
presents competitive market advantages among its competitors.
Costs of our solution
Looking at the Azure costs, we can see the total costs are lower than that of fully equipped
laboratories, and lower than having a local IT solution. The costs of our solution include the
Windows Azure Service and the cost for the end-devices.
Windows Azure hosting expenses would average at four (USD)/month/school, but
depending on how many experiments are performed and how widely it is deployed, can go much
lower.
For the end-devices, a various range can be considered. For this example, we can considered
the custom cubes with LCD displays on each side (IQubes), which can be used to simulating
experiments from many areas of study. We estimated a production cost of 15 (USD) per cube
and accordingly 90 (USD) per experiment kit containing simultaneously, and replace half of
them yearly due to breakdowns. Of course, the hardware device prices can range since we could
also have smartphones for example six cubes. We can estimate a school having 20 sets to allow
many pupils to perform the experiments.
That is quite a hefty difference! That money can be used for other advances, whilst not
compromising the education and development of the pupils.
Costs 1 school/10 years 25 schools/ 10 years TOTAL costs/10 years
Normal laboratory equipment 45,120 USD 1,128,000 USD 1,128,000 USD
Microsoft Azure costs 320 USD 8,000 USD 278,000 USD - *
Hardware platform* 9900 USD - * 247,500 USD - *
Difference 850,000 USD
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 100
Iqube games
Game 1: Numbers, objects and words
Goal: Match the numbers with words and pictures
Number of IQubes: 3
Description: The child must match correctly a mathematical number and certain images that
contain the same number of objects and with the word that spells that particular number. If you
rotate the first IQube forward or backwards the number increases or decreases. (Number are
shown as a mathematical symbol from 1-10). If you rotate the second IQube in any direction
images appear with random number of objects (From 1 to 10 objects per image). If you rotate the
third IQube in any direction a word will appear that spell a specific number (From 1 to 10). After
obtaining the matching number-image-word on the three IQubes the child must then join them
for verifying the result. If ok – the cube turns green. If not ok – the cube turns red.
three
Increase the number
3
five
Rotate to change the number
three
2
3
Cube 1
Cube 3
five2
Not ok
ok
Rotate to change the number of objects
Cube 2
Figure 65 Game example 1
Game 2: Numbers and colors Goal: To build an existing image using counting abilities and color matching.
Number of IQubes: 2
Description: On one face of the IQube an image is shown (ex. 5 red flowers). Using the second
IQube, the child must build the same image only by rotating the IQube. If the cube is rotated
forward, the number of flowers increases. If he rotates the IQube backwards, the number of
objects in the image decreases. If the IQube is rotated to the left or to the right, the color of the
objects in the image changes. For checking if the image is correct, the child must join the two
IQubes and the cube turns green if ok and red if not.
Cube 1 Cube 1 + Cube 2
Cube 1 + Cube 2
Cube 2Cube 2
Change color
Cube 2
Increase
number of
objects
Figure 66 Game example 2
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 101
Team games:
Every child has an IQube. Each of them is able to modify a certain characteristic of an
animal; the first child can modify the head, the second - torso, the third – legs, fourth – color, etc.
After joining all the cubes, final result is sent back to the eBox and the teacher evaluates the
result on the screen. Other team games may include puzzle games.
Other types of games:
Besides these main game types described, standalone games are also a possibility. Taking
full advantage of the accelerometer and the 6 LCD screens, games that involve fast and
continuous rotation of the cube in a certain way are easily integrated on this platform. These
types of games can be on a time based win condition and, when integrated with math and logic
skills can make the playing experience very educative, not just for young children, but also for
teenagers.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 102
Tools and software used
Hardware
AVR Dragon programmer
The AVR Dragon sets a new standard for low cost
development tools for 8-bit and 32-bit AVR devices with On
Chip Debug (OCD) capability. It can perform a symbolic
debug on all devices with OCD with SPI, JTAG, PDI (selected
devices), high voltage serial programming, parallel
programming, and aWire modes, and supports debugging using
SPI, JTAG, PDI interfaces.
Software
DipTrace
DipTrace is EDA software for creating schematic diagrams and printed circuit boards, a
complete state-of-the-art PCB Design System. It includes:
PCB Layout — PCB design with an easy-to-use manual routing tools, shape-based autorouter and auto-placer.
Schematic — Schematic Capture with multi-level hierarchy and export to PCB Layout, Spice or Netlist.
Component and Pattern Editors — allow you to make new parts and footprints.
Standard Libraries - include 100,000+ parts.
Realterm
Realterm is a terminal program specially designed for capturing, controlling and debugging
binary and other difficult data streams using the computer‘s serial interface. We used this
program to debug and communicate with the cubes while developing the application.
Audacity
Audacity is a free, easy-to-use and multilingual audio editor and recorder for Windows, Mac
OS X, GNU/Linux and other operating systems. It was used to convert, prelisten cut and
preprocess audio files berofe copying them to the SD Card
Fig. 67: AVR Dragon
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 103
AVR Studio® 5
Atmel® AVR Studio® 5 is the Integrated Development Environment (IDE) for developing
and debugging embedded Atmel AVR® applications. The AVR Studio 5 IDE gives you a
seamless and easy-to-use environment to write, build, and debug your C/C++ and assembler
code.
All the embbeded code is writen using this IDE.
CATIA V5R19
CATIA (Computer Aided Three-dimensional Interactive Application is a multi-platform
CAD/CAM/CAE commercial software suite developed by the French company Dassault
Systemes.
We used Catia to design and verify the devices we will construct.
Adobe Photoshop CS5
Adobe Photoshop is a graphics editor that is one of the most popular programs for image
manipulation. It is used widely by graphics professionals for all sorts of tasks including website
design. With our templates we often include a source .PSD file which is a photoshop document
that you will need Adobe Photoshop for. Being able to edit the source .PSD file will allow you to
make many different kinds of customizations to your template.
This program was used to create/moddify almost all graphical parts of this project.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 104
Implementation, testing and future work
We used the following methodologies for developing and testing our project:
Scrum - an iterative and incremental agile software development method for managing
software projects and product or application development.
Test-driven development (TDD) - a software development process that relies on the
repetition of a very short development cycle: first the developer writes a failing automated test
case that defines a desired improvement or new function, then produces code to pass that test and
finally refactors the new code to acceptable standards
Continuous integration (CI) - implements continuous processes of applying quality
control — small pieces of effort, applied frequently
Future improvements
Wireless charging
In the future we aim to recharge the battery of t he cubes using inductive coupling. For this
we have already begun experimenting with wireless energy transfer as described in the
experiment below:
Figure 68: Wireless Charging
L1: 66uH L2: 57,8uH; C1,C2 100 nF; S1 f=64.6 kHz
No inductive coupling Inductive coupling Efficiency
U S1 = 4V
U S1 = 1,3 V U D1 = 2.5 V
55%
Consumer product
In the future we plan to reconstruct the case completely. For this we made a new 3D model
of the case (fig 66). Also, we want to make a consumer version of our product, 6 cubes packed in
a nice transparent box, together with the charging platform (fig 67)
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 105
Figure 69: New 3D Model for new Case
Figure 70: Transparent carrying case with charging pad
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 106
Conclusions
With LiveX Learning Platform, we can reduce the inequality in universal education and help
the elementary schools from poor countries to deliver experiments in many different fields at a
much lower and affordable budged.
Many children will have the chance to interact in a more engaging and creative way with
science and understand it much better by experimenting with what they learn, rather than just
reading textbooks.
Teachers will have the possibility to create custom experiments and share them with other
schools by publishing them on our experiment marketplace, creating a community around our
solution.
We are fully aware that no simulation or virtual experiment can truly offer the same
experience as a real-life carried out experiment with real objects, but our platform is somewhere
between the pure virtual world and the practical world.
Moreover, our platform provides education to be accessible from any location, anywhere in
any time with countless possibilities of improvement.
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 107
Bibliography
Windows Communication Foundation Service
[EWSL09] ―School Laboratories in Developing Countries: Are They worth the Effort and
Expense?‖ - Erik W. Thuistrup – 1999, Science and Environment Education: Views from
Developing Countries, edited by Sylvia Ware
[VNLW01] The Laboratory in Science Education: Foundations or the Twenty-First
Century - AVI HOFSTEIN,VINCENT N. LUNETTA - 2003, Wiley InterScience
[VTYW02] ―Measuring Education Inequality: Gini Coefficients of Education‖ - Vinod
Thomas, Yan Wang, XiboFan – 2000, World Bank Institute
[RMPB01] Research Matters - to the Science Teacher, Patricia E. Blosser - 1990, Ohio
State University
[JGPT03] Critical Issue: Promoting Technology Use in Schools, by Jan Gahala, M.A.,
technical specialist in NCREL's Communications department.
[HSLWE01]http:/dawn.com/2011/03/06/school-labs-without-equipment/
[WGCSP02] http:/www.theregister.co.uk/2012/04/12/grand_challenges_science_policy/
[WIAM03] http: //www.theregister.co.uk/2012/04/13/india_adopts_microsoft_cloud/
[BAME02] Basis Education Movement (MEB), ―News about literacy – data from
Unesco, http://www.meb.org.br/noticias/unescoalfabetizacao;
[MUSM05] "Metro UI Design Principles". Stephane Massey. Retrieved 2012-02-17.
Windows Workflow Foundation interoperability with embedded devices
[DFWK01] Definition of workflow:
http://dictionary.reference.com/browse/workflow?s=t
[WWFH02] Windows Workflow Foundation homepage: http://msdn.microsoft.com/en-
us/netframework/aa663328.aspx
[HWWP03] Hardware Specifications for Windows Phone: http://msdn.microsoft.com/en-
us/library/ff637514(v=vs.92).aspx
[WWAL04] The Workflow Way: Understanding Windows Workflow Foundation - The
Base Activity Library: http://msdn.microsoft.com/library/dd851337.aspx
[WWKF05] The Workflow Way: Understanding Windows Workflow Foundation -
Kinds of Workflows: http://msdn.microsoft.com/library/dd851337.aspx
[PWFB06] Pro WF: Windows Workflow in .NET 4, Bruce Bukovics, Apress
[BWFC07] Beginning WF: Windows Workflow in .NET 4, Mark J. Collins, Apress
[PWPC08] – Programming Windows Phone 7 by Charles Petzold, Copyright © 2010 by
Microsoft Corporation
[IMVW09] Introduction to Model/View/ViewModel pattern for building WPF apps:
http://blogs.msdn.com/b/johngossman/archive/2005/10/08/478683.aspx
LiveX Learning Platform 2012
Facultatea de Automatica si Calculatoare Page 108
IQube: a cube for learning – embedded educational device
[DAIK01] Druin A, Inkpen K (2001) When are personal technologies for children?
Introduction to special issue personal technologies for children. Pers Technol 5:191–194
[PJHC03] Piaget J (1953) How children form mathematical concepts. Sci Am
189(5):74–79
[WRGE06] Research Group Embedded Interaction, Particle Display, Website
http://www.hcilab.org/projects/particles/particles-dis-play-add-on.htm
[WMLE07] Mind Lab‗s, http://www.mindlab.ro/metodologie/educational-process/.
[WXBE08] Digi International, "XBee/XBee-PRO RF Modules: Product Manual",
Digi International, Inc., USA, 2009, [Online: http://www.digi.com].
[DSDRA04] AVR1312: Using the XMEGA External Bus Interface
[ACCD01] Freescale Semiconductor Data Sheet: Technical Data , Document Number:
MMA8452Q, Rev 4.1, 08/2011
[MMCF06] - How to Use MMC/SDC (http://elm-chan.org/docs/mmc/mmc_e.html),
Update: November 3, 2010
[WAVE06] - WAVE PCM soundfile format
(https://ccrma.stanford.edu/courses/422/projects/WaveFormat/)