Software Engineer REST users rejoice! Conditional requests are here!
How would you increment an integer in the Realtime Database? It's not always as easy as just grabbing a value, adding one, and writing it back again. When multiple users are simultaneously reading, updating and writing the same value, some users' writes could get "lost" due to race conditions.
Let's say you're using the Realtime Database to store upvotes on videos, and two of your users want to upvote the "Silly Cat Video" which currently stands at ten upvotes. Each user reads the value '10' from the database. They then independently increment the value locally and write '11' back to the database. "Silly Cat Video" now only has eleven upvotes when it should have twelve.
How do we prevent this? On a mobile or web client, you could use transactions, which have long been supported in our realtime SDKs. But what if you're using the REST API?
Good news for you: we've just added conditional requests to the REST API! While conditional requests work differently than the realtime transactions, we can still use them to safely perform atomic operations.
Here's how you upvote that cat video the right way: Step 1: Request Data and ETag
Request:
# -i returns headers in the response, -H sets a header curl -i \ -H 'X-Firebase-ETag: true' \ 'https://<your-database>.firebaseio.com/videos/silly_cat_video/upvotes.json'
Response:
HTTP/1.1 200 OK ... ETag: ViJFJowpbyRvgGNPzPJdGeN+mCY= ...
10 // Value at the specified location
Step 2: Write the New Value
Request:
# -X sets the method (defaults to GET), -d provides data to the method curl -i \ -X PUT \ -d '11' \ -H 'if-match: ViJFJowpbyRvgGNPzPJdGeN+mCY=' \ 'https://<your-database>.firebaseio.com/videos/silly_cat_video/upvotes.json'
If no other users have incremented upvotes.json the PUT request will succeed. If another user has, the request will fail and the response will contain the new ETag and data. Use these to calculate a new upvotes value and retry the PUT request.
But wait, what does all this ETag and if-match stuff mean, and how do they work?
The Realtime Database's conditional REST requests use ETagsand the if-matchheader from the HTTP conditional requests standard. An ETag is a short unique identifier for data at a location: if the data changes, so does its ETag. if-match, when set to an ETagvalue, tells the database to only complete a request's operation if the data the request would overwrite matches the provided ETag. Together, these two features enable a Compare and Swap (CAS) paradigm. Compare and Swap uses the two step process we saw in our video upvote example: Step 1: Fetch the location you wish to update along with its ETag. Step 2: Issue a PUT request with new data in the body and the ETag of the old data in the if-match header. If the data at the location changes between Step 1 and Step 2, the database will not complete the operation and will return an error containing the new ETag and data. You can use these values to construct a new PUT request and retry from Step 2.
Sometimes the worst bugs to track down are the ones that seem to be impossible to reproduce. Or worse, inconsistent performance problems that can cause "Application Not Responding" errors in Android apps. No matter how much test code you write, these types of errors seem to have a way of sneaking into your app and causing problems for your users. However, with some clever use of Android's StrictModeAPI, alongside Firebase Test Lab for Android, you can find out about these problems before they reach production!
Over the years since Android was first available, a number of best practices have been observed for writing solid apps. For example, you shouldn't be performing blocking I/O on the main thread, and you shouldn't store references to Activity objects that are held after the Activity is destroyed. While it's likely that no one will force you to observe these practices (and you may never see a problem during development), enabling StrictMode in your app will let you know where you've made a mistake. These are called "policy violations", and you configure these policies with code in your app.
Personally, I don't want any of my code to cause any StrictMode violations. In fact, I'd like to consider them bugs that are just as serious as a regular crash, because they could crash my app in some cases! And, in fact, I can configure all StrictMode violations to crash the app. The Java code for that looks like this:
private static final StrictMode.ThreadPolicy FATAL_THREAD_POLICY = new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build();
private static final StrictMode.VmPolicy FATAL_VM_POLICY = new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build();
public static void enableFatalStrictMode() { if (shouldEnableFatalStrictMode()) { StrictMode.setThreadPolicy(FATAL_THREAD_POLICY); StrictMode.setVmPolicy(FATAL_VM_POLICY); } }
Here I have both a ThreadPolicy(for the main thread) and a VmPolicy(for the entire app) that look for all known violations, and will both log that violation and crash the app. Take a look through the javadoc for those builders to learn about all the situations they can catch.
Also notice that I'm enabling the policies conditionally. You probably don't want an accidental violation to crash on your users in production, so the method shouldEnableFatalStrictMode() typically looks like this, which activates it for your debug builds only:
What I'd like to do is take this further and have StrictMode crash my app also when it's running in Firebase Test Lab. This is handy because the crash will fail the test, and I'll get a report about that in the test results. For that to happen, I can change the method to query a system setting on the device that's unique to devices in Test Lab. Note that this requires an Android Context to obtain a ContentResolver:
Now, my instrumented tests (and the automated Robo test) will crash with any StrictMode violations, and I'll see those very clearly in my Test Lab report. Here's an app that crashed in Test Lab with a StrictMode violation during a Robo test:
This doesn't tell you exactly what happened, though. Digging into the logs in the test report, I see the details of the violation right before the crash:
D/StrictMode(6996): StrictMode policy violation; ~duration=80 ms: android.os.StrictMode$StrictModeDiskReadViolation: policy=327743 violation=2 D/StrictMode(6996): at android.os.StrictMode$AndroidBlockGuardPolicy.onReadFromDisk(StrictMode.java:1415) D/StrictMode(6996): at java.io.UnixFileSystem.checkAccess(UnixFileSystem.java:251) D/StrictMode(6996): at java.io.File.exists(File.java:807) D/StrictMode(6996): at android.app.ContextImpl.getDataDir(ContextImpl.java:2167) D/StrictMode(6996): at android.app.ContextImpl.getPreferencesDir(ContextImpl.java:498) D/StrictMode(6996): at android.app.ContextImpl.getSharedPreferencesPath(ContextImpl.java:692) D/StrictMode(6996): at android.app.ContextImpl.getSharedPreferences(ContextImpl.java:360) D/StrictMode(6996): at android.content.ContextWrapper.getSharedPreferences(ContextWrapper.java:167) D/StrictMode(6996): at com.google.firebasesandbox.MainActivity.onCreate(MainActivity.java:25) D/StrictMode(6996): at android.app.Activity.performCreate(Activity.java:6982) D/StrictMode(6996): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1213) D/StrictMode(6996): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2770) D/StrictMode(6996): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2892) D/StrictMode(6996): at android.app.ActivityThread.-wrap11(Unknown Source:0) D/StrictMode(6996): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1593) D/StrictMode(6996): at android.os.Handler.dispatchMessage(Handler.java:105) D/StrictMode(6996): at android.os.Looper.loop(Looper.java:164) D/StrictMode(6996): at android.app.ActivityThread.main(ActivityThread.java:6541) D/StrictMode(6996): at java.lang.reflect.Method.invoke(Native Method) D/StrictMode(6996): at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240) D/StrictMode(6996): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)
So we've discovered here that an activity's onCreate() called getSharedPreferences(), and this method ends up reading the device filesystem. Blocking the main thread like this could be the source of some jank in the app, so it's worth figuring out a good way to move that I/O to another thread.
Now we have a way to check for StrictMode violations during a test, but there's still a few more details to work out.
When should I enable StrictMode?
If you do this immediately when the app launches (in an Application subclass or a ContentProvider), there is a subtle bug in some versions of Android that may need to be worked around in order to avoid losing your StrictMode policies when the first Activity appears.
When should I disable StrictMode?
There might be some times when you know there is a violation, but you can't fix the code right away for whatever reason (for example, it's in a library you don't control). In that case, you might want to temporarily suspend the crashing behavior and simply log it instead. To temporarily suspend the crash, you can define a couple other non-fatal policies and swap them in where you know there's an issue to ignore:
private static final StrictMode.ThreadPolicy NONFATAL_THREAD_POLICY = new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .build();
private static final StrictMode.VmPolicy NONFATAL_VM_POLICY = new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .build();
public static void disableStrictModeFatality() { StrictMode.setThreadPolicy(NONFATAL_THREAD_POLICY); StrictMode.setVmPolicy(NONFATAL_VM_POLICY); }
Similarly, you may not want to track all the possible violations all the time. In that case, you might want to instruct the policy builder to only detect a subset of the potential issues.
Get out there and test your app!
While StrictMode violations are not always the worst thing for your users, I've always found it educational to enable StrictMode in order to find out where my app might be causing problems. In combination with Firebase Test lab, it's a great tool for maximizing the quality of your app.
Angular version 4.3 has been released. This is a minor release following ourannounced adoption of Semantic Versioning, meaning that it contains no breaking changes and that it is a drop-in replacement for 4.x.x.
What’s new?
We are introducing HttpClient, a smaller, easier to use, and more powerful library for making HTTP Requests. Learn more about it from our docs
New router life cycle events for Guards and Resolvers. Four new events: GuardsCheckStart, GuardsCheckEnd, ResolveStart, ResolveEnd join the existing set of life cycle event such as NavigationStart
Support for the emulated /deep/ CSS Selector (the Shadow-Piercing descendant combinator aka >>>) has been deprecated to match browser implementations and Chrome’s intent to remove. ::ng-deep has been added to provide a temporary workaround for developers currently using this feature.
For the complete list of features and bugfixes please see the changelog.
We have covered most of the topics in building a mobile application using Ionic 3 and Angular JS 4. Today’s topic is to pull in data for news feed from a server and display it on your website. This is suggestible post since it does proper verification at the backend based on token. All it does is, it will verify system token and user token at the backend and then pulls data using feed API url. I have also added an extra code to the previous post for login authentication with PHP Restful API for showing alert messages. Please do follow the below video and code for your understanding.
One of my favorite parts about working with the Cloud Functions for Firebase team is helping developers move logic from their mobile apps to a fully managed backend hosted by Firebase. With just a few lines of JavaScript code, they're able to unify logic that automatically takes action on changes to the contents of their Realtime Database. It's really fun to see what people build!
When Cloud Functionswas first announced at Cloud Next 2017, there was just one trigger available for all types of changes to the database. This trigger is specified using the onWrite() callback, and it was the code author's responsibility to figure out what sort of change occurred. For example, imagine your app has a chat room, and you want to use Firebase Cloud Messaging to send a notification to users in that room when a new message arrives. To implement that, you might write some code that looks like this:
exports.sendNotification = functions.database .ref("/messages/{messageId}").onWrite(event => { const dsnap = event.data // a DeltaSnapshot that describes the write if (dsnap.exists() && !dsnap.previous.exists()) { // This is a new message, not a change or a delete. // Send notifications with FCM... } })
Note that you have to check the DeltaSnapshotfrom the event object to see if new data now exists at the location of the write, and also if there was no prior data at that location. Why is this necessary? Because onWrite() will trigger on all changes to data under the matched location, including new messages, updated messages, and deleted messages. However, this function is only interested in newmessages, so it has to make sure the write is not an update or a delete. If there was an update or delete, this function would still get triggered and return immediately. This extra execution costs money, and we know you'd rather not be billed for a function that doesn't do useful work.
Today, it gets better
The good news is that these extra checks are no longer necessary with Cloud Functions database triggers! Starting with the firebase-functions module version 0.5.9, there are now three new types of database triggers you can write: onCreate(), onUpdate(), and onDelete(). These triggers are aware of the type of change that was made, and only run in response to the type of change you desire. So, now you can write the above function like this:
Here, you don't have to worry about this function getting triggered again for any subsequent updates to the data at the same location. Not only is this easier to read, it costs you less to operate.
Note that onWrite() hasn't gone away. You can still keep using it with your functions for as long as you like.
Avoid infinite loops with onWrite() and onUpdate()
If you've previously used onWrite() to add or change data at a location, you're aware that the changes you made to that data during onWrite() would trigger a second invocation of onWrite() (because all writes count as writes, am I right?).
Consider this function that updates the lastUpdated property of a message after it's written:
This seems OK at first, but there's something very important missing. Since this function writes back to the same location where it was triggered, it will effectively trigger another call to onWrite(). You can see here that this will cause an infinite loop of writes, so it needs some way to bail out of the second invocation.
It turns out that onUpdate() function implementations share this concern. One solution in this specific case could be to check the existing value of lastUpdated and bail out if it's not more than 30 seconds older than the current date. So, if we want to rewrite this function using onUpdate(), it could look like this:
exports.lastUpdate = functions.database .ref("/messages/{messageId}").onUpdate(event => { const msg = event.data.val() const now = new Date().getTime() if (msg.lastUpdated > now - (30*1000)) { return } msg.lastUpdated = now return event.data.adminRef.set(msg) })
This function now defends against infinite loops with a little extra logic.
To start using these new triggers, be sure to update your firebase-functions module to 0.5.9. And for more information about these updates, check out the documentation right here.
Building a great game is tough. You have to make it easy to pick up, challenging, a little bit addicting, and, of course, fun. Not to mention the technical effort involved in designing all the infrastructure that will power your blockbuster game. Our goal with Firebase is to take care of that last piece, so that you can focus on all of the elements that make your game unique.
The team hard at work building AppShip3000. Soldering irons are fun!
How it works
AppShip3000 is a 3 person, collaborative, rocketship-flying, trivia game. The players work together to control a rocket ship that is flying upwards and quickly running out of fuel. Asteroids rain down from the top of the screen, while the crew answers Firebase and Google Cloud trivia to reach higher heights.
The catch is that players can only see either the question or one of the multiple choice answers, never both. They have to talk (or yell) to each other to figure out which player has the right answer before time runs out. Similarly, one player sees asteroids before the others and has to shout out where they'll be, so the entire team can move the rocket. All players have to move the joystick in that same direction to successfully avoid losing fuel from an asteroid collision.
Building with Firebase
Now, let's take a look at how Firebase powers AppShip3000. First off, we use Firebase Authenticationfor the sign-in experience. A player opens a progressive web app (PWA) on her phone and logs in using her Google account. Creation of an account is an Authentication Trigger for a Cloud Function that generates a custom button sequence (or launch code). This launch code is used to associate the account with the game controller for the duration of the game, so that when the game finishes, all stats are automatically associated with her account.
The finished product at Google I/O 2017!
When a game finishes, Realtime Database is used to immediately update the leaderboard in the PWA. At I/O, we also had giant screens reflecting the leaderboards. Realtime Database is also used to manage the trivia questions in the game. This allows for on-the-fly updates to questions that can be written after one game and pushed to production in time for the next game.
We also generate an image of the rocket's flight path after each game and store it in Cloud Storage for Firebase. This action triggers another Cloud Function that sends a notification to the player, using Firebase Cloud Messaging. The notification lets the player know about the souvenir image and prompts her to share it with friends.
Getting Started
We had a blast building AppShip3000. Using Firebase to handle most of the infrastructure allowed us to focus on building a unique game that (we think) turned out to be a lot of fun to play. If you're interested in using Firebase in your own game, check our our C++ and Unity documentation. And to learn more about how Firebase and Google Cloud were used to manage the data pipeline of AppShip3000, check out our I/O talk. Can't wait to see what you build!
In the six months since our last post about Angular Material, the team been hard at work building cool new components and applying a heaping of polish. Some of the highlights include:
Datepicker, expansion panel, and accordion components
Improved support for @angular/platform-server
Configurable scrolling strategies for Overlays
Strict-null checks and no-unused-params compliance
We're excited to announce today's release of the brand new Component Dev Kit, and the initial version of the data-table component with Angular Material beta 8.
Component Dev Kit
Over the course of working on Angular Material, an important goal has been crafting a general, reusable foundation upon which components can be built. We've reached the point now where we want to start sharing this foundation with everyone as a standalone package: the Angular CDK.
The goal of the CDK is to give developers more tools to build awesome components for the web. This will be especially useful for projects that want to take advantage of the features of Angular Material without adopting the Material Design visual language.
The first beta release of the CDK is now available on npm as @angular/cdk. This initial release includes features pertaining to accessibility, text directionality, platform detection, and dynamic component instantiation. We'll be continuing to move additional code from Angular Material into CDK with subsequent releases.
To get started building components using the CDK, install it from our npm repository and take a look at our documentation on GitHub for usage information.
Data-table
By far the most requested feature for Angular Material, the data-table is now making its debut. We're now releasing two table implementations, <md-table> as part of @angular/material, and <cdk-table> as part of @angular/cdk. Our styled implementation of our Material Design <md-table> is based on the functionality provided by the <cdk-table> component from the new CDK.
The <cdk-table> is an unopinionated, customizable data-table with a fully-templated API, dynamic columns, and an accessible DOM structure. This component acts as the core upon which anyone can build their own tailored data-table experience.
@angular/material also builds on the data-table functionality, offering <md-paginator> and mdSortdirectives. These directives provide a UI for pagination and sorting in-line with the Material Design guidelines without being tightly coupled to the data-table itself.
For more information on how to get started with the data-table, see our guide, or check out the rest of the documentation on material.angular.io.
This is just the first taste of what we have in store for data-tables. You can watch the development of upcoming features like sticky headers, sticky columns, incremental row rendering, colspans, and more on our GitHub Pulse! You can also see the team's planned work on GitHub.
We hope you enjoy our new features and we look forward to hearing the community's feedback on GitHub and Twitter!
Last April, we announcedthe general availability of the Firebase Admin SDK for Python. The initial release of this SDK supported two important features related to Firebase Authentication: minting custom tokens, and verifying ID tokens. Now, we are excited to announce that database support is available in the Firebase Admin SDK for Python starting from version 2.1.0.
Due to the way it's implemented, there are several notable differences between this API and the database APIs found in our other Admin SDKs (Node.js and Java). The most prominent of these differences is the lack of support for realtime event listeners. The Python Admin SDK currently does not provide a way to add event listeners to a database reference in order to receive realtime update notifications. Instead, all data retrieval operations are provided as blocking methods. However, despite these differences there's a lot that can be achieved using this API.
The database module of the Python Admin SDK facilitates both basic data manipulation operations and advanced queries. To begin interacting with the database from a Python environment, initialize the SDK with the Realtime Database URL:
import firebase_admin from firebase_admin import credentials
Then obtain a database reference from the db module of the SDK. Database references expose common database operations as Python methods (get(), set(), push(), update() and delete()):
from firebase_admin import db
root = db.reference() # Add a new user under /users. new_user = root.child('users').push({ 'name' : 'Mary Anning', 'since' : 1700 })
# Update a child attribute of the new user. new_user.update({'since' : 1799})
# Obtain a new reference to the user, and retrieve child data. # Result will be made available as a Python dict. mary = db.reference('users/{0}'.format(new_user.key)).get() print 'Name:', mary['name'] print 'Since:', mary['since']
In the Firebase Realtime Database, all data values are stored as JSON. Note how the Python Admin SDK seamlessly converts between JSON and Python's native data types.
To execute an advanced query, call one of the order_by_* methods available on the database reference. This returns a query object, which can be used to specify additional parameters. You can use this API to execute limit queries and range queries against your data, and retrieve sorted results.
from firebase_admin import db
dinos = db.reference('dinosaurs')
# Retrieve the five tallest dinosaurs in the database sorted by height. # 'result' will be a sorted data structure (list or OrderedDict). result = dinos.order_by_child('height').limit_to_last(5).get()
# Retrieve the 5 shortest dinosaurs that are taller than 2m. result = dinos.order_by_child('height').start_at(2).limit_to_first(5).get()
# Retrieve the score entries whose values are between 50 and 60. result = db.reference('scores').order_by_value() \ .start_at(50).end_at(60).get()
Take a look at the Admin SDK documentation for more information about this new API. Also check out our Github repo, and help us further improve the Admin SDK by reporting issues and contributing patches. In fact, it was your continuing feedback that motivated us to build and release this API in such a short period. Happy coding with Firebase!
Testing your application is a great way to help maximize its quality, and many of you know that Firebase Test Lab for Android has some useful tools for testing Android apps. If you're the type of engineer who likes to maximize test coverage by writing instrumented tests (and regular unit tests), you can send those to Test Lab for execution. Even if you don't like writing tests, you have some options. You can record instrumented tests by interacting with your app using Espresso Test Recorder in Android Studio. And there's almost no effort required at all to run a Robo test that automatically crawls your app.
These tests are helpful for data-driven apps because there are robust test frameworks that understand how to navigate the Android platform widgets used to receive input and display data on screen. However, most games don't work with platform widgets. Games typically take over the screen using their own UI elements, and provide their own touch controls. As a result, it's extremely difficult to write instrumented tests for testing games, and Robo test won't know how to navigate the game at all.
To help deal with these challenges with testing games, the Test Lab team has come up with a way for game developers to test their games effectively across many of the devices that Test Lab offers. It's a new type of test called a Game Loop Test, and it's available today in beta.
How does a Game Loop test work?
If you've seen arcade video games operate, you know that they're always showing something on screen, typically some automated demo of the game, called "attract mode". With Firebase Test Lab, game developers can now use this concept of attract mode to construct test scenarios, and Test Lab will arrange for those scenarios to be invoked in sequence. This gives developers the opportunity to test a wide variety of game levels and situations in a single test run on all the devices provided by Test Lab. If there are any problems with a scenario, you'll find out which ones are problematic, so you can focus your efforts on improving that specific case.
Even better, Test Lab now provides performance data with every report. Of particular interest to game developers is a graph of the rendered frame rate over time throughout the test, tied to a video of the device display. This helps you quickly identify the parts of your game that aren't rendering at an acceptable FPS.
In addition to FPS, there are other performance metrics available for all apps. You'll be able to see your app's CPU utilization, memory usage, and network ingress and egress. Notice how you can jump directly to the point in the video where you're observing performance problems.
If you're a game developer, now is a great time to check out Firebase Test Lab for Android to see what it can do to help the quality of your game.
