Save My Planet Virtual Performance

Here is a 1000-cubic-feet eXtended Reality environment and the second-in-the-world live interactive virtual performance using Mixed Reality Technology, a remarkable achievement by 41 people.

Watch an ocean whale fly overhead. The effect is as grand as an ocean amidst the total immersive LED Screen experience with AR and life-tracking technologies.

Dive into people’s emotions that are deeper than the seas of space. We have enabled the audience’s real-time “touch” with the artist and their connection across a long distance to reach, fascinate, and interact on a new level. https://freckledsky.com/works/savemyplanet-interactive-virtual-performance/

Saltation - Laser + Projection mapping Audiovisual Performance

Projection mapping using a laser projector of 30,000 pixels to analyze the characteristics of the exterior wall of the building. The projected image is given to the space to give a formative beauty of the dotted surface, and a high output. The laser device extends the planar screen into a three-dimensional space for a visual effect. Expressing content as a synesthetic experience, technology and art, future and culture, experimentation. Progressive, and other keywords are expressed metaphorically through points, lines, faces, and light. To this end, images, lasers, and music are linked together and interact organically.

Audiovisual Dance Performance

Audiovisual Dance Performance, presented in March 2015 at UT Austin, , Oscar G. Brockett Theatre. Section where a performer holding a brain computer interface (EEG) and a gyroscope interacts with the visual space and sound in real time. The EEG creates a direct link between the performer who activates and modulate the audiovisual signal in real time by concentration levels of facial expression and additional minimalistic physical movement. With the aid of a gyroscope the visualizations follow the orientation of the body, creating the illusion of infinite space on the floor.

Web Performance 2.0

In March 2012, Guy Podjarny ran a test comparing the performance of hundreds of shiny new responsive websites across four different screen resolutions. The results were very dissapointing. (1)

Two years into the rise of Responsive Web Design, after every imaginable sort of designer and developer had jumped into the train, it took a test to almost rock the theory to its foundations.

Guy proved that almost every known responsive site was overweight.

We’ve made the internet in our image… ObeseJason Grigsby

But, most importantly, every mobile user was receiving the same kilobyte overload as a desktop user.

The community had different reactions to the fact. Some claimed Responsive design wasn’t the ultimate solution, perhaps not mature enough for the challenges web designers face today.

Thankfully, the Web community can always count on a number of people who will grab the bull by the horns and turn the situation around. Modern Web gurus likeBrad Frost, Luke Wroblewski or Christian Heilmann saw opportunity where others shouted crisis and managed to turn the problem to the community’s advantage.

If your website is 15MB, it’s not HTML5, it’s stupidChristian Heilmann

Web performance has traditionally been built around (no offence) developer-exclusive jargon.

Terms like GZIPing, uglifying, minifying, DNS Lookup, file-concatenation… This obscure words push designers out of the ecuation.

Smart people in the community, though, have since realized that the problem has a deeper root. It really doesn’t matter if you optimize or compress an ultra-high-res image, if your plan is to hide it from a mobile user and still make him download it.

Good performance is good designBrad Frost (2)

To achieve truly lightweight sites, performance shouldn’t only be a concern. It should be treated as a design feature.

To them, performance is like any other issue. Sites that overcome it are the ones who acknowledged it from the start. And the ones that overlook it are the ones that suffer it in the end.

Performance is about respect. Respect your users and they will come back.Brad Frost

The Why

It’s not only that you want users to have a good experience. If that was the case you could easily swap performance for a more important concern in your agenda.

Page Abandonment

Research shows 57% of users will leave your site if it takes more than 3 seconds to load. (3)

Google, page speed & SEO

Back from the spring of 2010, Google takes speed as a ranking factor. (4)

The impact is not major for average-speed sites, but if the page falls behind a certain threshold, it will be punished by the company’s search algorythm.

This proves speed not only is a concern when talking about User Experience. Be aware of loading speed if your site is to be ranked well among the competition.

Bandwidth considerations

Back in the day, people used to talk about the very abstract concept of ‘Mobile Context’. Google´s famous theory breaks mobile users down to three types (5):

Repetitive Now

People that use their phone to stay up to date with ongoing, repetitive changes (sports scores, Facebook feeds or stock market)

Bored Now

Users that put their phone out while waiting for something to happen.

Urgent Now

Pretty self explanatory

Sounds assumable, right?

Well, the truth is there is no truth about this. There is no 'mobile context'. People will use their phone not only when they are walking in the street, travelling by train or relaxing in their home. They do everything at the same time!

Phones follow people everywhere, so people use them anywhere.

Mobile context is important, but first we need to figure out what the heck it is.Tim Kadlec

Luke Wroblewski shows some really interesting stats: (6)

Where are people using mobile devices?

• 84% at home
• 80% during miscellaneous downtime throughout the day
• 76% waiting in lines of waiting for appointments
• 69% while shopping
• 64% at work
• 62% while watching TV (alt. study claims 84%)
• 47% during commute in to work

As new situations emerge, as new markets and different habits rise, mobile context will change. We can safely assume that the concept of mobile context will always be on the move, until people stop using mobile phones.

This leads us to keep an eye on bandwidth. There is only one scenario where you can serve users an obese website and get away with it: serve it to their macbook pros, while they are at home in UK or the USA with a full bandwidth.

But the rest of possible situations, which are a great many, have to be covered aswell. These include the seemingly endless stream of devices poured every day into the market, which people use to visit websites.

You don’t get to decide which devices access your site, your users doKaren McGrane

They include the countries which didn’t have that many smartphones a few years ago, but are ruthlessly getting ahead.

If your stuff, if your content, if your information, if your products, if your services are not available on mobile, they don’t exist for these peopleKaren McGrane

But more importantly, they include all the places people will be at when using your site. So you have to watch all bandwidths. It’s not only inhabitants of poor areas of the world clearly don’t have the same data-speed coming their way. People will try to access a site at work, with a 100mb/s connection; at home, with 2 to 30mb/s and also with 3G, and also with 4G, and also with a data plan, etc., etc.

To put it bluntly, Responsive design is not anymore about screen sizes but about different scenarios, so the solutions must be flexible, adaptable and thought out top to bottom.

And How?

Well, glad you asked.

We said before not to look at performance as a bunch of automated tasks run server side that help with an already doomed site. There are ways to undertake this concerns and turn them into a competitive advantage.

What to avoid

Guy Podjarny cites three main reasons for the number of bloated responsive websites we see out there: (1)

Download and Hide

Assets are still downloaded, but hidden.

Download and Shrink

High-res desktop-level images are downloaded, and shrinked to fit the users screen

Excess DOM

There is no way to avoid browsers parsing and processing all areas of the DOM, including the hidden ones

A preemptive approach

There’s a great deal of information out there about why websites keep failing to surpass expectations in performance. But what most people come to say is something like ‘Be responsible from the start’. All techniques I’m going to cover have been around for a while. To me the interesting part comes in how they mix and intertwine, covering each other’s flaws and combining their strenghts. It is now, deep in the mobile explosion that they show how powerful they are.

Progressive Enhancement…

…is all about providing a web experience reduced to the essential and take it from there.

A couple of years ago this theory was taken mostly from a browser point of view. With emerging technologies like HTML5, CSS3, jQuery and so on, the web makers had kind of forgotten about their users. Quite a big percentage of them were getting an incomplete form of their site, relying a bit too much on this shiny new tech.

Now that Webkit engines and Firefox and others have taken over much of the market share, the problem is the enormous quantity of devices with browsers that don’t have the capabilities of the brand new iPhone or Samsung. Again, Progressive Enhancement is the only approach which takes care of these forgotten players first and leave the shine for the ones that can take it.

Mobile First Development

Back in 2009, Luke Wroblewski proposed designing mobile first for three reasons: (7)

• Mobile is exploding
• Mobile forces you to focus (allowing you to get rid of the clutter that stems from having too much screen real estate)
• Mobile extends your capabilities (with technology like GPS, geolocation, multi-touch gestures, accelerometer, cameras…)

Since then, Web design has been rapidly shifting to this approach. Along the way not only designers, also many developers, have pointed out that building mobile first gives you an edge over desktop development, very related to Luke W’s second point. Progressive Enhancement and Mobile First Development have suffered a fusion of sorts. Devs start building for mobile and progressively enhance from there, taking larger screen space as an enhancement over a mobile core foundation.

Jordan Moore makes a good summary of the reasons(8). He argues that, since 'we can't safely bet on connection speed', 'The 'responsible Web designer' would build for the lowest point of entry - a mobile-first approach, assuming for the slowest connection speed and building up from there to larger breakpoints for faster connections'. In the future, we will be able to rely on solid bandwidth detection, but for now it is a good idea to take it as a concern and try not to take any steps in the wrong direction.

To sum up:

Code the site for the lowest resolution and possibilities that you care about. Make true use of Progressive Enhancement from the start. Build extra functionality, enhanced visuals and interaction when it can be used.

RESS: REsponsive Web design + Server Side components

To many people, Responsive design had kind of an essential short-coming. It relied mainly in screen width detection.

As more and more types of devices emerge, hybrid devices like touch screen laptops and so on, feature detection has become essential for Responsive design. Libraries that provide it, mainly Modernizr, have bloomed and are now used on most projects. They help devs valuate whether the client’s browser supports certain functionality and provide it accordingly. But many times it’s tricky to rely on browsers, because ‘they’ will say they support features when, really, ‘they’ do whatever they want. Support for new features is usually partial.

RESS was born to provide a solution. Like Mobile First, the term was coined by Luke Wroblewski in 2011 (9). It relies on detecting the user’s device type, evaluating it and providing an experience taylored for it. To do this, there are heavy tools out there, like WURFL, DeviceAtlas or lighter ones like Browser Gem, that read the user agent string and start from there.

Evaluating the device type has other advantages. It allows devs to serve different templates depending on the user’s device. Say you are building a ver large site, and you want your mobile navigation to be a simplified one, that doesn´t take half as much space as the desktop one. You could either play with content, showing and hiding stuff, moving divs around with JavaScript, or you could have different templates for mobile and desktop screens and have the server decide which one to serve.

This gives Responsive design an edge over Mdot sites. Mdot’s only advantage until RESS came along was providing an experience specific for mobile devices.

The BBC (very smart people, with millions of readers across the globe and a big responsibility toward their users) talk about how RESS and Progressive Enhancement could work as one and only. They call their approach Cut the Mustard! (10). It consists of creating a core experience that will work on every device you can imagine. After that, they evaluate the device on the server and they decide whether or not it ‘Cuts the mustard’. If it does, a progressively enhanced experience is handed out. But again, if it doesn’t, the user can still access the core content.

Conditional Loading

Mobile users want to see our menu hours and delivery number. Desktop users definitely want this 1mb png of someone smiling at saladMat ‘Wilto’ Marquis

Let’s take a couple of points of view into account:

• Mobile users want THE content, as much as desktop users.

If your content is accessible from a URL, it will be accessed by mobile devices.Brad Frost

• Mobile forces you to focus. There are some constraints designers have to embrace to serve the same content, like bandwidth and lesser screen size.

Also refered to as ‘Agressive Enhancement’, this development technique allows designers to focus on the core content and progressively enhance it for bigger screens. It provides basic access to certain content that can later be injected on the page when space becomes available.

It might be more accurate to think of conditional loading as a content-first approach. You don’t have the luxury of sidebars or multiple columns to fill up with content that’s just nice to have rather than essentialJeremy Keith

Use excellent tools like MatchMedia, that mimics CSS behaviour evaluating screen size in JavaScript.

Lazy Loading

Image and user heavy sites that need to be optimized for mobile, like Facebook, Twitter or Pinterest, make use of Lazy Loading to provide a better experience. When you first load the page, a number of posts is loaded. When you scroll down, the designer assumes it is because you want to browse through even more content, so it is injected in the page via Ajax. This makes the page load much faster by avoiding DOM excess.

Setting a performance budget

Tim Kadlec argues that setting a maximum page weight and being always aware of it is the ultimate way to keep page load down (11). ‘Set your goals and stick to them’. Steve Souders mentions three options to choose from, if you fall over your budget:

• Optimize an existing feature or asset
• Remove an existing feature or asset
• Don’t add a new feature of asset.

To me this sounds a bit radical, but it makes a point of closely following the overall performance of a site over time and with each new feature.

Let’s get Technical!

There are certain methods to achieve speed, that work in a more technical and less conceptual level.

Image Techniques

Images constitute around 60% of websites. If you are serving mobile users with unknown bandwidth connections desktop-sized images, you are basically dooming your site to poor performance

The trick to overcome this is to serve different versions of images, depending on screen size or type. You would serve a small image to a mobile phone and a high-res one to a desktop. You would serve a double-sized image to a Hi-DPI device.

Responsive Images

Designers and developers all over the world have been fighting to get responsive images into the HTML specification. Mat ‘Wilto’ Marquis is one of the most outspoken. The battle is not yet won, but there are a number of solutions that rely on JavaScript to achieve a desired result. Scott Jehl, also from the Filament Group, wrote a plugin that mimics the markup proposed by the community and works like a charm: PictureFill

Compressive Images

Daan Jobsis, a dutch designer, found a very strange phenomenon when compressing images with Photoshop (12). He proved the following: Take an image, double its size (200%), compress it to 25% or less its original quality, resize it back in the browser(100%). The image will not only be lighter in size but already optimized for HiDPI screens, since its pixel density is already doubled.

The only observed problem is that the browser might have a hard time painting a double-sized image back to its original size (if it has to do it a hundred times, like in image-heavy sites), so a little bit of testing is required to see if this is the optimal solution.

Vectors VS Bitmaps

SVG images are the way to go at the time. They are completely scalable, so they will perform better in any screen. Providing fallback is very easy through Modernizr.

Icon Fonts

Technically they are vector based images, only served as a font. As Chris Coyier puts it, ‘Icon Fonts are Awesome because’:

• You can easily resize
• You can easily change the color
• You can easily shadow their shape
• They will work in IE6, unlike transparent PNGs
• You can do everything you can do with images
• You can do anything you would do with typography

HiDPI images

Dave Bushell wrote recently a very interesting article with some thinking on HiDPI images (12). He argues that, even if today we have the possibility of serving iPhones, iPads and other modern devices with images that will fulfill their screen capabilities, it is still too soon to assume a site is not going to get crippled by doing it.

Does a fast connection and high pixel density mean users even want higher quality? Not likely on mobile data plans.Dave Bushell

The point is to do it but do it sensibly, considering the case before jumping into 4x images.

What’s next

Google recently developed a new image format, WebP. It provides lossless and lossy compression for Web images, resulting in files 3x smaller, compared with PNG.

There are simple, lightweight JavaScript libraries that convert to and from WebP available today. Considering the impact of Google’s latest tools, it’s probably a good idea to start experimenting today in order to handle an image-heavy site .

Asset Loading

Load assets carefully and in order. Controlling this aspect provides a big advantage, by allowing the page to render the basic content and enhance it afterwards.

CSS, Images

Controlled loading through media queries, conditional or lazy loading and responsive / compressive image techniques

JavaScript

Make use of HTML5 functionality, like async or defer. There are also loading helpers like RequireJS that can handle loading and dependencies.

Advertising, Social Widgets or any third party assets

Just inject after load.

Old-school Performance Techniques

They have been around for a while, but are still as relevant today.

Reduce the number of HTTP Requests

To achieve this devs have to think resource by resource, but here are a number of guidelines:

• Concatenate all CSS files or make use of CSS Preprocessors to compile them into one file.
• Unify all JS Plugins under the same file and always load them in the footer, unless they really need to block the rendering of the page (if you load Typekit fonts in the footer, you will get the famous FOUT or ‘Flash of Unstyled Text’).
• If you must use PNG images, use sprites. They unify all images in one and make use of CSS to cut the pieces. There are a number of online solutions to do this.
• Make use of the data URI scheme where possible, that allows you to include images as inline data, getting rid of some more HTTP requests.

Reduce the number of Bytes

Uglify, minify every Script or CSS file you call from the page. Set your server up to allow GZIP compression and expansion and GZIP every asset.

In Summary

The importance of Web performance has been slightly overlooked since the birth of Responsive design.

Designers and developers have been focusing on how to solve the Responsive puzzle and, along their way, a new multi-bandwidth, multi-device, multi-location Web is starting to come into focus.

To be prepared for tomorrow’s problems, we have to include performance as an essential consideration, as the Desktop-centered Web is disappearing before our eyes. The mobile user is hastier and readier and won’t jump through hoops to get the content, and since more and more sites spring every day, being fast will mean being ahead.

---

Reference

1. Performance Implications of Responsive DesignGuy Podjarny

2. Performance as DesignBrad Frost

3. Website abandonment happens after 3 seconds

4. How Page Load Speed Impacts SEO And User ExperienceSpencer Yao

5. Organizing MobileLuke Wroblewski

6. When & Where Are People Using Mobile Devices?Luke Wroblewski

7. Mobile FirstLuke Wroblewski

8. Responsible Considerations For Responsive Web DesignJordan Moore

9. RESS: Responsive Design + Server Side ComponentsLuke Wroblewski

10. Cutting the mustardBBC´s Responsive News

11. Setting a Performance BudgetTim Kadlec

12. Retina Revolutie Follow UpDaan Jobsis

13. The Raster Image ParadoxDave Bushell

Follow the discussion 

Source

Responsive Web Design Summit talk, 'Measuring Web Performance'

On Tuesday, April 16, 2013, I was lucky enough to give the opening talk on the web performance day of RWD Summit presented by Environments for Humans. This is the short description for the talk:

Today, a web page can be delivered to desktop computers, televisions, or handheld devices like tablets or phones. While a technique like responsive design helps ensure that our web sites look good across that spectrum of devices we may forget that we need to make sure that our web sites also perform well across that same spectrum. More and more of our users are shifting their Internet usage to these more varied platforms and connection speeds with some moving entirely to mobile Internet.

In this session we’ll look at the tools that can help you understand, measure and improve the web performance of your web sites and applications. The talk will also discuss how new server-side techniques might help us optimize our front-end performance. Finally, since the best way to test is to have devices in your hand, we’ll discuss some tips for getting your hands on them cheaply.

This presentation builds upon Dave’s “Optimization for Mobile” chapter in Smashing Magazine’s ”The Mobile Book.”

Measuring Web Performance from Dave Olsen

Article: Source

iOS 6 for HTML5 developers, a big step forward

Article: Source

The new main version of the Apple’s iOS is with us, along with the new iPhone 5 and the iPod Touch fifth generation. As every big change, lot of new stuff is available for HTML5 developers and -as always- no much official information is available.

QUICK REVIEW

I’m going to divide this post in two parts: iPhone 5 and iOS 6 new stuff.

On iPhone 5:

• New screen size
• New simulator
• What you need to do
• Problems

New features on iOS 6:

• File uploads and camera access with Media Capture and File API
• Web Audio API
• Smart App Banners for native app integration
• CSS 3 Filters
• CSS 3 Cross Fade
• CSS Partial Image support
• Full screen support
• Animation Timing API
• Multi-resolution image support
• Passbook coupons and passes delivery
• Storage APIs and web app changes
• Web View changes for native web apps
• Debugging with Remote Web Inspector
• Faster JavaScript engine and other news

IPHONE 5

The new iPhone 5 -along with the iPod Touch 5^th generation- has only one big change in terms of web development: screen resolution. These devices have a wide 4″ screen, WDVGA (Wide Double VGA) 640×1136pixels, 326 DPI -Retina Display as Apple called it. These devices have the same width as iPhone 4/4S but 176 more pixels-height on portrait mode.

NEW SIMULATOR

iOS Simulator on Xcode 4 includes iPhone 5 emulation

The new Xcode 4 (available on the Mac AppStore) includes the updated iOS Simulator. The new version has three options for the iPhone simulation:

• iPhone: iPhone 3GS, iPod Touch 1^st-3^rd generation
• iPhone Retina 3.5″: iPhone 4, iPhone 4S, iPod Touch 4^th generation
• iPhone Retina 4″: iPhone 5, iPod Touch 5^th generation

The new simulator also includes the new Maps application replacing Google Maps by default and Passbook.

WHAT YOU NEED TO DO FOR THE NEW DEVICES

Usually, if your website/app is optimized for vertical scrolling, you should not have any problem. Same viewport, icons and techniques for iPhone 4/4S should work properly.  Remember, when updating the iOS, you are also updating the Web View: that means that all the native web apps -such as PhoneGap/Apache Cordova apps- and pseudo-browsers such as Google Chrome for iOS are also updated. However if your solution is height-dependent, then you may have a problem.Just look at the following example of the Google Maps website on iPhone 4 and iPhone 5. As it is talking the height as a constant, the status bar is not hidden and there is a white bar at the bottom.

Be careful if you’ve designed for an specific height as Google Maps. As you can see (right caption is from iPhone 5) there is a white bottom bar and the URL bar can’t be hidden as there is no enough content.

If you are using Responsive Web Design you should not have too much trouble as usually, RWD techniques are using the width and not the height for conditionals.

DEVICE DETECTION

At the time of this writing there are no iPhone 5 on the street yet. However, as far as every test I could check, there is no way to detect iPhone 5 server-side. The user agent only specifies an iPhone with iOS 6, and the same exact user agent is being used for iPhone 4S with iOS 6 and iPhone 5.

Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) 
AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25

Therefore, the only way to detect the presence of a 4″ iPhone device is to use JavaScript and/or media queries, client-side. If you need to know server-side, you can plant a cookie from client-side for next load. Remember that these devices have 1136 pixels height, but in terms of CSS pixels (independent resolution pixels) we are talking about 568 pixels-heightas these devices have a pixel ratio of 2.

isPhone4inches = (window.screen.height==568);

Using CSS Media Queries and Responsive Web Design techniques,we can detect the iPhone 5 using:

@media (device-height: 568px) and (-webkit-min-device-pixel-ratio: 2) {
/* iPhone 5 or iPod Touch 5th generation */
}


HOME SCREEN WEBAPPS

For Home Screen webapps the problem seems important. I’ve reported the problem while in NDA without any answer from Apple yet.

Basically, when you add a website to the Home Screen that supports apple-mobile-web-app-capable meta tag, your webapp works only in iPhone 3.5″ emulation mode (it’s not taking the whole height) as you can see in the following example from the Financial Times webapp.

The letterbox black bars you see here are not a problem in the image. That is how a full-screen webapp is being launched by default on iPhone 5 and the new iPod Touch.

While it’s a good idea to not add more height to a webapp if the OS is not sure about it’s compatibility on a wider screen, as far as I could test, there is no way to define that our webapp is 4″ compatible. I’ve tried as many combinations as I could think about, and if you provide an apple-touch-startup-image of 640×1096, the iPhone 5 takes your splash screen but it’s being resized to 640×920, at least in the Simulator for the GM compilation (almost the final version).

UPDATE 9/20: Solution found, thanks to some guys that were pointing to some solutions I;ve found the trick. As weird as it sounds, you need to forget about the viewport with width=device-width or width=320. If you don’t provide a viewport, it will work properly. The same if you use other properties than width; if you don’t want your viewport to be the default 980px, the way to do it is:

<meta name="viewport" content="initial-scale=1.0">

Even if you use a viewport for an specific size different than 320 letterbox will not be present.

<meta name="viewport" content="width=320.1">

Instead of changing all your viewports right now, the following script will make the trick changing it dynamically:
if (window.screen.height==568) { // iPhone 4"
document.querySelector("meta[name=viewport]").content="width=320.1";
}

The startup image has nothing to do with the letterbox as some developers were reporting. Of course, if you want to provide your launch startup image it has to be 640×1096 and you can use media queries to use different images on different devices. Some reports were saying that you need to name the launch image as in native apps “Default-568h@2x.png” but it’s not true. You can name it however you want. The sizes attribute is completely ignored.

You can use media queries to provide different startup images:

<link href="startup-568h.png" rel="apple-touch-startup-image" media="(device-height: 568px)">
<link href="startup.png" rel="apple-touch-startup-image" sizes="640x920" media="(device-height: 480px)">

If you want to provide an alternative version for low resolution devices then you can use the -webkit-device-pixel-ratio conditional too. If you are wondering why 568px and not 1136px remember that we are using CSS pixels and in these devices the pixel ratio is 2.

The trick is the viewport. Why? I don’t really know. For me, it’s just a bug. But it’s the only solution I’ve found so far.

The other problem is with Home Screen icons that you are already have before buying your new device. iTunes will install the shortcut icon again from your backup and it’s not clear if we are going to have a way to upgrade the compatibility. Even if you change the viewport, if the icon is already installed before the change you will get the letterbox.

IOS 6 AND HTML5 DEVELOPMENT

iOS 6 is available as a free update for every iOS 5 device but not the iPad first generation so we will see this version browsing the web really soon and the iPad market is being fragmented for the first time. The following findings are useful for all iOS devices that are talking the iOS 6 upgrade. As always -and unfortunately- Apple is giving us just partial and incomplete updates on what’s new on Safari and I -as always- enter the hard work of digging into the DOM and other tricks to find new compatibility.

FILE MANAGEMENT

Finally! Safari for iOS 6 supports a file upload input type and with HTML Media Capture partial support.

A simple file upload as the following, will ask the user for a file, from the Camera or Gallery as you can see in the figure. I really like how Safari is showing you screenshots instead of a temporary filename after selecting your image.

<label>Single file</label>
<input type="file">

We can also request multiple files using the HTML5 new boolean attribute. In this case, the user can’t use the camera as a source.
<label>Multiple files</label>
<input type="file" multiple>

We can access the camera and gallery using file uploads

There is no way to force the camera, as using capture=”camcorder”. However, we can specify if we want to capture images or videos only, using the accept attribute.

<input type=file accept="video/*">
<input type=file accept="image/*">


There is no support for other kind of files, such as audio, Pages documents or PDFs. There is no support for getUserMedia for live camera streaming.

What you can do with the image or video after selected?

• Send it using multipart POST form action (old-fashion upload mechanism)
• Use XMLHttpRequest 2 to upload it using AJAX (even with progress support)
• Use the File API that is available on iOS 6 that allows JavaScript to read the bytes directly and manipulate the file client side. There is a good example of this API in action on HTML5Rocks.

WEB AUDIO API

HTML5 game developers should be happy! Web Audio API appears on a mobile browser for the first time. This API allow us to process and synthesize audio on JavaScript. If you have never played with some low level audio, the API may seems a little weird, but after a while is not so hard to understand. Again, HTML5Rocks has a great article to begin with the Audio API.

More information and news on the API on http://www.html5audio.org

SMART APP BANNERS

Website or native app? If we have both, now we can join efforts and connect our website with our native app. With Smart App Banners, Safari can show a banner when the current website has an associated native app. The banner will show a “INSTALL” button if the user doesn’t have the app installed or a “VIEW” button to open it if installed. We can also send arguments from the web to the native app. The case is to open the native app on the same content that the user were seeing on the web site.

To define a Smart App Banner we need to create a meta tag, with name=”apple-itunes-app”. We first need to go and search for the app we have on iTunes Link Maker and take the app ID from there.

<meta name="apple-itunes-app" content="app-id=9999999">

We can provide a string value for arguments using app-argument and if we participate in the iTunes Affiliate program we can also add affiliate-data in the same meta tag.

<meta name="apple-itunes-app" content="app-id=9999999, app-argument=xxxxxx">
<meta name="apple-itunes-app" content="app-id=9999999, app-argument=xxxxxx, affiliate-data=partnerId=99&siteID=XXXX">

The banner takes 156 pixels (312 on hi-dpi devices) at the top until the user click on the bottom or on the close button and your website will get the full height for you. It acts like a DOM object at the top of your HTML but it’s not really on the DOM. On iPad -and more on landscape- it seems a little space-wasting.

With Smart App Banners, the browser will automatically invite the user to install or open a native app

During some seconds, the banners shows a “loading” animation while the system is verifying that the app suggested is valid to the current user’s device and App Store. If it’s not valid, the banner hides automatically; for example, it’s an iPad-only app and you are browsing with an iPhone or the app is available only on the german App Store and your account is in US.

CSS 3 FILTERS

CSS 3 Filters a set of image operations (filters) that we can apply using CSS functions, such as grayscale, blur, drop-shadow, brightness and other effects. These functions will be applied before the content is rendered on screen. We can use multiple filters using spaces (similar to transforms).

You can try a nice demo here. A quick example of how it looks like:

-webkit-filter: blur(5px) grayscale (.5) opacity(0.66) hue-rotate(100deg);

CSS 3 CROSS-FADE

iOS 6 start supporting some of the new CSS Image Values  standard, including the cross-fade function. With this function, we can apply two images on the same place with different levels of opacity and it can even part of a transition or animation.

Quick example:
background-image: -webkit-cross-fade(url("logo1.png"), url("logo2.png"), 50%);

FULL SCREEN IN SAFARI

Besides the chrome-less home screen meta tag, now the iPhone and iPod Touch (not the iPad) supports a full-screen mode when in landscape. This is perfect for immersive experiences such as games or multimedia apps. There is no way to force full-screen mode and it needs to be launched by the user (last icon on the toolbar). However, we can invite the user to move to landscape first and press on the full-screen icon to activate our app. If we mix this with some touch event handling we can hide the URL bar and provide a good interface until the user escape from the full-screen.

Fullscreen navigation on iPhone and iPod Touch

You will always find two or three overlay buttons at the bottom that your design should be aware of, that’s the back button, the optional forward button and the cancel full-screen.

You can use the onresize event to detect if the user is changing to full-screen while in landscape.

ANIMATION TIMING API

Game developers, again, you are lucky. iOS 6 supports Animation Timing API, also known as requestAnimationFrame, a new way to manage JavaScript-based animations. It’s webkit prefixed, and for a nice demo and more detailed explanation check this post from Paul Irish.

CSS IMAGE SET

This is not part of any standard group yet. It’s a new image function, called image-set receiving a group or images with conditions to be applied. The only compatible conditions right now seems to be 1x and 2x for low density devices and high density devices. With this new function we don’t need to use media queries to define different images for different resolutions. The working syntax is:
-webkit-image-set(url(low.png) 1x, url(hi.jpg) 2x)
It’s working on CSS, such as a background-image. I couldn’t make it work on the HTML side, for the src attribute on an img element or the new proposed pictureelement. With this new syntax we can have more clear multi-resolution image definition, as we don’t need to use media queries and background-size values.

PASSBOOK COUPONS AND PASSES DELIVERY

Passbook is a new app in iOS that works as a virtual container for all your passes, ticket, discount coupons, loyalty cards and gift cards. As a web developer you may want to serve the user with a discount coupon, a ticket to an event, an e-ticket for your next flight or a loyalty card.

Apple allow websites to deliver this kind of passes from a website without the need of a native app.

To deliver the pass on your website you just need to use the MIME type application/vnd.apple.pkpass or send it through email

Apple provides a tool that you can install on your server to package and sign customized passes on the fly that may include current user information

To pass file is just a JSON meta-data file and a couple of images. We need to package the file and sign it. Unfortunately, to sign the pass we need a signature from Apple and that means that the web developer needs an iOS Developer Program account ($99/year). If you receive the pass already signed, you can just insert it on your own site.

One of the great features of passes is that once installed you can provide some web services on your end and through Push Notification Services, the operating system will call your web services to update the information on the pass.

More information at developer.apple.com/passbook

STORAGE APIS AND WEBAPP UPDATES

No, there is no new storage API available. There is no support for IndexedDB yet. However, there are some changes you should take in consideration:

• Application Cache limit was increased to 25Mb.
• Chromeless webapps (using the apple-mobile-web-app-capable meta tag) now have their own storage sandbox. That means that even if they are served from the same domain, the web app from the Home Screen will have its own persistent Local and SQL Storage. Even if you install the icon many times, every icon will have its own sandbox. While this is good news for apps, it may be also a problem on some apps if you are passing information from the website to the home screen widgets through storage.
  Credits for this finding to George Henne on his post.
• There is a new undocumented meta tag that can be used on any website (having the apple-mobile-web-app-capable meta tag or not) that allow us to define adifferent title for the Home Screen icon. As you may know, by default Safari takes the document’s title and crop it to 13 characters. Now we can define an alternative title for the Home Screen using:

<meta name="apple-mobile-web-app-title" content="My App Name">

I’ve also found a meta tag called apple-mobile-web-app-orientations accepting the possible values portrait, portrait-upside-down, landscape-right,landscape-left, portrait-any. Unfortunately, I couldn’t make it work. If you have any luck feel free to comment here.

WEB VIEW UPDATES

On Web View (pseudobrowsers, PhoneGap/Cordova apps, embedded browsers) JavaScript now runs 3.3x slower (or let’s say that Nitro engine on Safari and Web apps is 3.3x faster). Be careful about the 3.3x, that is just the different of running SunSpider on the same device, on Web View and Safari. However, SunSpider is not covering all possible kind of apps and your total rendering time is not just JavaScript, so this doesn’t mean that your app runs 3.3x slower.

We can find some other good news:

• Remote Web Inspector for webapp debugging
• A new supressesIncrementalRendering Boolean attribute that can eliminate the partial rendering mechanism. I believe this feature is useful to reduce the perception of loading a web page instead of being an app.
• A new WebKitStoreWebDataForBackup info.plist Boolean feature where we can define that we want localStorage and Web SQL databases to be stored in a place to be backed up, such as in iCloud. This problem has appeared in iOS 5.01, now it’s solved
• Changes in the developer agreement: it seems that the restriction of using only the native WebView to parse HTML and JS has gone. It will be good if someone from Apple can confirm this. The only mention to the internal WebKit engine is that it’s the only engine capable of downloading and execute new code, while in the same app expected behavior; that’s the anti-Chrome statement. You can use your own engine but only if you are not downloading the code from the web. This may be opening a door… such as delivering our own engine, for example, with WebGL support.

REMOTE DEBUGGING

I’m keeping this topic at the end. Because it’s a huge change for web developers. For the first time, Safari on iOS includes an official Remote Web Inspector. Therefore tools, such as iWebInspector or Weinre will become obsolete since this version. The Remote Debugger works with the Simulator and with real devices via USB connection only.

To start a remote inspection session you need to use Safari 6 for desktop. Here comes the bad news: you can only debug your webapp on a Mac desktop computer. It was a silent change, but Safari for Windows is not available anymore, so it’s stuck in 5.x. Therefore, only with a Mac OS computer you can make web debugging session on your iOS devices (at least officially for now).

For security reasons, you need to first enable the Web Inspector from Settings > Safari > Advanced. The new Inspector means that the old JavaScript console is not available anymore.

You can start a debugging session with:

• A safari window on your iOS device or simulator
• A chrome-less webapp installed on your iOS device or simulator
• A native app using a Web View, such as Apache Cordova/PhoneGap apps.

When talking about native apps, you can only inspect apps that were installed in the device by Xcode (your own apps). Therefore, there is no way to inspect Google Chrome on iOS websites for example.

If you are used to the Webkit Inspector -Safari 5 or Chrome-, you are going to see a completely redesign version of the inspector in Safari 6 based on Xcode native development UI. You will be lost for a while understanding the new UI. With the inspector session, you can:

• See and make live changes on your HTML and CSS
• Access your storages: cookies, local storage, session storage and SQL databases
• Profile your webapp, including performance reports for Network requests, Layout & Rendering and JavaScript and events. This is a big step in terms of performance tools.
• Search on your DOM
• See all the warning and errors in one place
• Manage your workers (threads)
• Manage JavaScript breakpoints, and define Uncaught exception breakpoint.
• Access the console and execute JavaScript
• Debug your JavaScript code
• Touch to inspect: There is a little hand icon inside the inspector that allows you to touch on your device and find that DOM element on the inspector.

Well done Apple, we were waiting for this on iOS for long time. Apache Cordova users should be also happy with this feature.

OTHER SMALLER UPDATES

• Apple claims to have a faster JavaScript engine. And it seems to be true. On the SunSpider test I’m receiving 20% improvement on JavaScript performance on the same device with iOS 5.1 and iOS 6.
•  Google Maps is not available anymore on iOS 6; Now http://maps.google.com redirects to the Google Maps website and not the native app. there fore there is a new URL scheme, maps, that will open the native new Maps applications. The syntax is maps:?q=<query> and query can be just a search or latitud and longitude separated by comma. To initiate a route navigation, the parameters are: maps:?saddr=<source>&daddr=<destination>.
•  XHR2: Now the XMLHttpRequestProgressEvent is supported
• The autocomplete attribute of the input is officially in the DOM
• Mutation Observers from DOM4 are now implemented. You can catch a change in the DOM using the WebKitMutationObserver constructor
• Safari no longer always creates hardware-accelerated layers for elements with the -webkit-transform: preserve-3d option. We should stop using it for performance techniques.
• Selection API through window.selection
• <keygen> element
• Canvas update: Now the createImageData has one parameter and now there are two new functions that the name suggest to be prepared to provide High Resolution images webkitGetImageDataHD and webkitPutImageDataHD.
• Updates to SVG processor and event constructors
• New CSS viewport related measures: vh (viewport height) vw (viewport width) and vmin (minimum between vw and vh)
• CSS3 Exclusions and CSS Regions were available on beta 1 but they were removed from the final version. It’s a shame although they were too new and not mature enough.
• iCloud tabs. You can synchronize your tabs between all your devices, including Macs, iPhones and iPads. So the same URL will be distributed through all devices. Be careful on your mobile web architecture!

WHAT WE ARE STILL WAITING FOR

There are a couple of things that we still need to wait for a next version, such as:

• IndexedDB
• FileSystem API
• Performance Timing API
• WebRTC and getUserMedia
• WebGL -still there and still disabled-
• Orientation Lock API for gaming/inmersive apps
• Integration with Facebook and Twitter Accounts native APIs, so we can use current OS user’s credentials automatically instead of forcing the user to log in again.

FINAL THOUGHTS

Safari on iOS 6 is a big step for HTML5 developers; debugging tools, new APIs, better JavaScript performance. However, I must say that Apple is still forgetting about documentation updates and properly communication with web developers. There are almost no answers on the discussion forum, no updates on the Safari documentation (some docs are really too old right now).