Developer Advocate If you've been working with Cloud Functions for Firebase, you've probably wondered how you could speed up the development of your functions. This is possible for HTTPS type functions using the firebase serve in the Firebase CLI, but this wasn't an option for other types of functions. Now, local testing of all of your functions is easy with the Firebase CLI. If you want to try out your code before you deploy it to Cloud Functions, you can do that with the Cloud Functions shell in the Firebase CLI starting at version 3.11.0 or later.
Here's how it works, in a nutshell. We'll use a Realtime Database trigger as an example.
Imagine you have an existing project with a single function in it called makeUppercase. It doesn't have to be deployed yet, just defined in your index.js:
This onCreate database trigger runs when a new message is pushed under /messages with a child called original, and writes back to that message a new child called uppercased with the original value capitalized.
Now, if you can kick off the emulator shell from your command line using the Firebase CLI:
$ cd your_project_dir $ firebase experimental:functions:shell
Then, you'll see something like this:
i functions: Preparing to emulate functions. ✔ functions: makeUppercase firebase>
That firebase prompt is waiting there for you to issue some commands to invoke your makeUppercase function. The documentation for testing database triggers says that you can use the following syntax to invoke the function with incoming data to describe the event:
makeUppercase('foo')
This emulates the trigger of an event that would be generated when a new message object is pushed under /messages that has a child named original with the string value "foo". When you run this command in the shell, it will generate some output at the console like this:
info: User function triggered, starting execution info: Uppercasing pushId1 foo info: Execution took 892 ms, user function completed successfully
Notice that the console log in the function is printed, and it shows that the database path wildcard pushId was automatically assigned the value pushId1 for you. Very convenient! But you can still specify the wildcard values yourself, if you prefer:
After emulating this function, if you look inside the database, you should also see the results of the function on display, with /messages/{pushId}/uppercased set to the uppercased string string value "FOO".
You can simulate any database event this way (onCreate, onDelete, onUpdate, onWrite). Be sure to read the docs to learn how to invoke them each correctly.
The Cloud Functions shell is currently an experimental offering, and as such, you may experience some rough edges. If you encounter a problem, please let us know by filing a bug report. You can also talk to other Cloud Functions users on the Firebase Slack in the #functions channel.
Some tips for using the shell
Typing the function invocation each time can be kind of a pain, so be sure to take advantage of the fact that you can navigate and repurpose your invocation history much like you would your shell's command line using the arrow keys.
Also note that the shell is actually a full node REPL that you can use to execute arbitrary JavaScript code and use special REPL commands and keys. This can be useful for scripting some of your test code.
Since you can execute arbitrary code, you can also dynamically load and execute code from other files using the require() function that you're probably already familiar with.
And lastly, if you're like me, and you prefer to use a programmer's editor such as VS Code to write your all JavaScript, you can easily emulate functions by sending code you want to run to the Firebase CLI. This command will run test code from a file redirected through standard input:
Developer Advocate If you've browsed the web at all, you've probably seen some sites that ask you to prove you're a human by presenting a reCAPTCHA challenge. For example, if you try to use the goo.gl URL shortener, it won't let you shorten a link until you satisfy the reCAPTCHA, which looks like this:
Web site engineers do this to protect their site from spam and abuse from bots, while allowing legitimate human use. Why is protection needed? Maybe you have some backend code that's expensive in time and storage and you only want actual users of your web to access it.
If you have a web site, you can also use reCAPTCHA to protect its services. And, if you're building your site with Firebase Hosting, it's pretty easy to get it integrated with the help of Cloud Functions for Firebase to provide a secure, scalable backend to verify the completion of the reCAPTCHA.
In this blog post, I'll walk you through a few steps that will get you to a very basic integration that you can extend later for your own site. For this walkthrough, I'm assuming you already have some experience with web development, the Firebase console, and the Firebase CLI.
1. Create a Firebase project in the console
Navigate to the Firebase console and create a new project. There's no need to add billing to this project - you can experiment fully without providing a credit card. Once you create the project, there's nothing else you need to do in the console.
2. Set up a directory for your project code
Using the Firebase CLI, make sure you're logged in with the same Google account that you used to create the project:
$ firebase login
Now, create a root project directory and initialize it:
$ mkdir my_project $ cd my_project $ firebase init
When running firebase init, be sure to select both hosting and functions. When you're asked to choose a project, select the one you just created earlier. Take the defaults for every other prompt. You'll end up with a directory structure that contains a public folder for web content, and a functions folder for your backend code.
For the Cloud Functions backend, we'll need a couple modules from npm to help verify the reCAPTCHA. The reCAPTCHA API requires you to make an HTTP request for verification from your backend, and you can do that with the request and request-promise modules. Pull them into your project like this:
$ cd functions $ npm install request request-promise
Your package.json file should now show those two new modules in addition to firebase-functions and firebase-admin.
3. Test web deployment
Make sure you can deploy web content by running this deploy command:
$ firebase deploy --only hosting
When this finishes, you'll be given the public URL to your new web site, which will look something like this:
where your-project is the unique id that was given to your project at the time it was created in the console. If you paste the Hosting URL into your browser, you should see a page that says "Firebase Hosting Setup Complete".
4. Get a reCAPTCHA API Key
reCAPTCHA requires a couple API keys for operation, one for the web client and one for the server API. You can get those from the reCAPTCHA admin panel, so navigate there. Create a new site and give it a name. Select "reCAPTCHA V2". For domains, put the full hostname of your Firebase Hosting site name (e.g. "your-project.firebaseapp.com").
After you register, you'll be given a Site key and a Secret key. The Site key will be used in your frontend HTML, and the Secret key will be used in your backend hosted by Cloud Functions.
5. Add a page with a reCAPTCHA
Now we'll add a new HTML page to display the reCAPTCHA. In the public directory in your project, add a new HTML file called recaptcha.html to display the reCAPTCHA. Simply copy and paste the following content directly into that new file:
Notice in the body there is a div with the class "g-recaptcha". The first thing you should do here is copy your reCAPTCHA site key into the div's data-sitekey attribute value. This div will get automatically transformed into a reCAPTCHA UI after the first script at the top is loaded. You can read more about that here in the docs.
You can see it right away if you firebase deploy again, then navigate to /recaptcha.html under your Hosting URL. Don't bother dealing with the reCAPTCHA yet, because we still need some backend code to complete the verification!
The JavaScript code in this page defines two functions dataCallbackand dataExpiredCallback. These are referenced in the div, and provide callbacks for the reCAPTCHA to tell you when the reCAPTCHA has been satisfied, or if the user took too long to proceed.
The important thing to note in dataCallback is that it redirects the browser to another URL in the site with the path /checkRecaptcha, and pass it a parameter named response. This response string is generated by reCAPTCHA and looks like a random collection of characters.
The path /checkRecaptcha in your web site obviously doesn't exist yet, so we need to create a Cloud Function to validate the response string it's going to receive.
6. Create a Cloud Function to verify the reCAPTCHA response
In the functions directory in your project, edit the existing index.js file. This has some sample code, but you can delete it. In its place, paste the following JavaScript code:
const functions = require('firebase-functions') const rp = require('request-promise')
The first thing you should do here is paste your reCAPTCHA secret key from the registration site in place of "PASTE_YOUR_SECRET_CODE_HERE".
(Astute readers may note that the reCAPTCHA API endpoint host is "recaptcha.google.com", while the docs say "www.google.com". This is OK! You have to use recaptcha.google.com as shown in order to make the call on the Spark plan, because that host has been whitelisted for outgoing traffic from Cloud Functions.)
This code defines an HTTPS function that, when triggered, will make another HTTPS request (using the request-promise module) to the reCAPTCHA API in order to verify the response that was received in the query string. Notice that there are three cases with three different responses to the client. Either:
The reCAPTCHA verifies successfully (the user is human)
The reCAPTCHA fails (could be a robot)
The API call fails altogether
It's important to send a response to the client in all cases, otherwise the function will time out with an error message in the Firebase console log.
To deploy this new function (and the web content at the same time) run the following command:
$ firebase deploy
You'll notice in the output that the function is assigned its own URL, which looks something like this:
This is clearly a different host than the one with your web content. However, what we really want instead is for the function to be referenced through your web host at a URL that looks like this:
What you've done here is add a new section for rewrites, and you can read more about those in the docs. Specifically what this does is allow access to the URL path /checkRecaptcha invoke the function called checkRecaptcha that you pasted into your functions/index.js file.
Remember that the JavaScript code in recaptcha.html redirects to this path when the reCAPTCHA is satisfied by the user, so this effectively sends to user to the function after they complete the reCAPTCHA.
Now do one final deploy to send everything to Firebase:
$ firebase deploy
8. Test the reCAPTCHA!
Navigate to /recaptcha.html under your hosting URL, then solve the reCAPTCHA. It may ask you to identify some cars or roads in a set of pictures. Once you've satisfied the reCAPTCHA with your humanity, the JavaScript in your HTML should redirect you to your function, which verifies with the server that you're indeed human, and you should see the message "You're good to go, human."
This example of how to use reCAPTCHA with Cloud Functions for Firebase is much more simple than what you'd probably do in your own web site. You have several options for how to send the reCAPTCHA response to your function, and you'd obviously want to provide something more useful than a message to the user. But this should get you started protecting your web content from abuse from bots.
One of my favorite parts about working with the Cloud Functions for Firebase team is helping developers move logic from their mobile apps to a fully managed backend hosted by Firebase. With just a few lines of JavaScript code, they're able to unify logic that automatically takes action on changes to the contents of their Realtime Database. It's really fun to see what people build!
When Cloud Functionswas first announced at Cloud Next 2017, there was just one trigger available for all types of changes to the database. This trigger is specified using the onWrite() callback, and it was the code author's responsibility to figure out what sort of change occurred. For example, imagine your app has a chat room, and you want to use Firebase Cloud Messaging to send a notification to users in that room when a new message arrives. To implement that, you might write some code that looks like this:
exports.sendNotification = functions.database .ref("/messages/{messageId}").onWrite(event => { const dsnap = event.data // a DeltaSnapshot that describes the write if (dsnap.exists() && !dsnap.previous.exists()) { // This is a new message, not a change or a delete. // Send notifications with FCM... } })
Note that you have to check the DeltaSnapshotfrom the event object to see if new data now exists at the location of the write, and also if there was no prior data at that location. Why is this necessary? Because onWrite() will trigger on all changes to data under the matched location, including new messages, updated messages, and deleted messages. However, this function is only interested in newmessages, so it has to make sure the write is not an update or a delete. If there was an update or delete, this function would still get triggered and return immediately. This extra execution costs money, and we know you'd rather not be billed for a function that doesn't do useful work.
Today, it gets better
The good news is that these extra checks are no longer necessary with Cloud Functions database triggers! Starting with the firebase-functions module version 0.5.9, there are now three new types of database triggers you can write: onCreate(), onUpdate(), and onDelete(). These triggers are aware of the type of change that was made, and only run in response to the type of change you desire. So, now you can write the above function like this:
Here, you don't have to worry about this function getting triggered again for any subsequent updates to the data at the same location. Not only is this easier to read, it costs you less to operate.
Note that onWrite() hasn't gone away. You can still keep using it with your functions for as long as you like.
Avoid infinite loops with onWrite() and onUpdate()
If you've previously used onWrite() to add or change data at a location, you're aware that the changes you made to that data during onWrite() would trigger a second invocation of onWrite() (because all writes count as writes, am I right?).
Consider this function that updates the lastUpdated property of a message after it's written:
This seems OK at first, but there's something very important missing. Since this function writes back to the same location where it was triggered, it will effectively trigger another call to onWrite(). You can see here that this will cause an infinite loop of writes, so it needs some way to bail out of the second invocation.
It turns out that onUpdate() function implementations share this concern. One solution in this specific case could be to check the existing value of lastUpdated and bail out if it's not more than 30 seconds older than the current date. So, if we want to rewrite this function using onUpdate(), it could look like this:
exports.lastUpdate = functions.database .ref("/messages/{messageId}").onUpdate(event => { const msg = event.data.val() const now = new Date().getTime() if (msg.lastUpdated > now - (30*1000)) { return } msg.lastUpdated = now return event.data.adminRef.set(msg) })
This function now defends against infinite loops with a little extra logic.
To start using these new triggers, be sure to update your firebase-functions module to 0.5.9. And for more information about these updates, check out the documentation right here.
Many Firebase developers focus on their apps alone, usually for Android, iOS, or the Web (or all three!). If you're like me, and you spend most of your time on the client side, it can be a hassle to also deal with any backend components. Now that we have Cloud Functions for Firebase, it's super easy and fun to write and deploy backend code in JavaScript without having to manage servers. Personally, my comfort zone is with Android. So, to get acquainted with Cloud Functions, I spent some time brushing up on my JavaScript and learning node.js, which Cloud Functions uses to run backend code.
Enter the Promise
As part of that period of ramping up, I learned a new way of managing asynchronous work in JavaScript. If you've had experience working with JavaScript in the browser or with Cloud Functions, you may already be familiar with a class called Promise. A promise is a very common way to handle async work. The closest concept for Android developers is the Task API that you sometimes use when dealing with Firebase APIs.
A promise lets you respond to a unit of work that's executing asynchronously, such as a database write or a network request. Cloud Functions understand promises; if you want a function to stay alive during async work, you can do this by returning a promise from the function (except for HTTP/S triggers, which require a response sent to the client). When you return a promise, be sure to return a promise that is "resolved" after all asynchronous work performed in that function is fully complete. Many developers have found this video from the Firebase channel on YouTube very helpful for understanding how promises work with Cloud Functions:
In the video, Jen describes two main ways of dealing with promises. First, there's using the then() method on a promise to chain items of work in sequence, where the next item of work depends on the results from the previous item. Second, there's Promise.all() which creates a new promise that resolves after multiple items of work happening in parallel are all complete. Between these two mechanisms, you can manage most work that normally occurs in a function. When the work is complete, you return the final promise from the function so that the Cloud Functions environment knows when to clean up.
ECONNRESET got you down?
For simple functions, this works out pretty easily. But many of you (including myself) have discovered that complicated functions can be really difficult to manage when there's a lot of work (with lots of promises) going on. One of the symptoms of mismanaged promises is seeing the following error in the function log in the Firebase console:
Error: read ECONNRESET
This error could mean a few different things. In some cases it could be a bug in a client library. In other cases, it's a symptom of a common mistake when dealing with promises.
If a function is terminated before its network connections are finished, that means those open connections could be forced to close, leaving an ECONNRESET in the log. Here's a situation when that can occur.
Imagine you want to start three items of async work, and the work is kicked off in a function called doSomeAsync(), which returns a promise that's resolved upon completion, and you return the promise from the third task like this:
Returning a Promise from the function is good. But Cloud Functions needs a promise that resolves when all of the work is fully complete. If we don't have a guarantee that the last item of work will always finish after the other two, then Cloud Functions may clean up the function prematurely, causing problems. Instead, we need to combine all of the promises from all of the work into a new promise that resolves only after all of the others resolve, like this:
If you're not tracking every single item of asynchronous work that you kick off in a function, you could see the dreaded ECONNRESET error in your log, and things may not work the way you expect. It really can take some diligence to keep all your promises!
If you want to see an example of a more complicated function, I have an open source tic-tac-toe web game that I wrote for a session I gave at Google I/O '17 that talks about how it was built entirely with Firebase.
Three years ago, we launched Firebase Hosting to make it easier for developers to deliver fast, engaging experiences on the web. Two months ago, we launched the beta of Cloud Functions for Firebase to let developers write custom backend logic without having to worry about servers or infrastructure. More recently at Google I/O, we brought Firebase Hosting and Cloud Functions together to provide a flexible set of tools to build Progressive Web Apps with world-class scale and performance.
You can connect an HTTPS Cloud Function to your Firebase Hosting app by adding a rewrite to the firebase.json configuration for your project:
Once connected, matching requests seamlessly proxy to your Cloud Function (in the example above, a function named myFunction). This one-line change enables exciting new capabilities for Firebase Hosting users including:
Server-Side Rendering. Until now, Firebase Hosting has only served static content. Now you can serve dynamic content using industry-standard libraries like Express, while still leveraging a lightning-fast global CDN cache.
Custom APIs. With Cloud Functions and Firebase Hosting, you can create custom API endpoints on your own domain, avoiding the overhead of cross-origin requests. Plus, with a few extra lines of code you can authorize your endpoints through Firebase Auth.
User-Generated Web Content. For the first time, Firebase Hosting can generate content compatible with AMP, Twitter Cards, and more without requiring a deploy. This means your Firebase Hosting apps can be more shareable and more searchable than ever before.
For Cloud Functions users, you can now run functions on an SSL-secured custom domain and get powerful caching to avoid unnecessary executions.
In apps that allow users to post public content - for instance in forums, social networks and blogging platforms - there is always a risk that inappropriate content could get published. In this post, we'll look at ways you can automatically moderate offensive content in your Firebase app using Cloud Functions.
The most commonly used strategy to moderate content is "reactive moderation". Typically, you'll add a link allowing users to report inappropriate content so that you can manually review and take down the content that does comply with your house rules. You can better prevent offensive content from being publicly visible and complement your reactive moderation by adding automated moderation mechanisms. Let's see how you can easily add automatic checks for offensive content in text and photos published by users on your Firebase apps using Cloud Functions.
We'll perform two types of automatic content moderation:
Text moderation where we'll remove swearwords and all shouting (e.g. "SHOUTING!!!").
Image moderation where we'll blur images that contain either adult or violent content.
Automatic moderation, by nature, needs to be performed in a trusted environment (i.e. not on the client), so Cloud Functions for Firebase is a great, natural fit for this. Two functions will be needed to perform the two types of moderation.
Text Moderation
The text moderation will be performed by a Firebase Realtime Database triggered function named moderator. When a user adds a new comment or post to the Realtime Database, a function is triggered which uses the bad-words npm package to remove swear words.We'll then use the capitalize-sentencenpm package to fix the case of messages that contain too many uppercase letters (which typically meaning users are shouting). The final step will be to write back the moderated message to the Realtime Database.
To illustrate this we'll use a simple data structure that represents a list of messages that have been written by users of a chat room. These are made of an object with a text attribute that gets added to the /messages list:
/functions-project-12345 /messages /key-123456 text: "This is my first message!" /key-123457 text: "IN THIS MESSAGE I AM SHOUTING!!!"
Once the function has run on the newly added messages, we'll add two attributes: sanitized which is true when message has been verified by our moderation function and moderated which is trueif it was detected that the message contained offensive content and was modified. For instance, after the function runs on the two sample messages above we should get:
/functions-project-12345 /messages /key-123456 text: "This is my first message!", sanitized: true, moderated: false /key-123457 text: "In this message I am shouting." sanitized: true, moderated: true
Our moderator function will be triggered every time there is a write to one of the messages. We set this up by using the functions.database().path('/messages/{messageId}').onWrite(...)trigger rule. We'll moderate the message and write back the moderated message into the Realtime Database:
if (message && !message.sanitized) { // Retrieved the message values. console.log('Retrieved message content: ', message);
// Run moderation checks on on the message and moderate if needed. const moderatedMessage = moderateMessage(message.text);
// Update the Firebase DB with checked message. console.log('Message has been moderated. Saving to DB: ', moderatedMessage); return event.data.adminRef.update({ text: moderatedMessage, sanitized: true, moderated: message.text !== moderatedMessage }); } });
In the moderateMessage function, we'll first check if the user is shouting and if so fix the case of the sentence and then remove all bad words using the bad-words package filter.
function moderateMessage(message) { // Re-capitalize if the user is Shouting. if (isShouting(message)) { console.log('User is shouting. Fixing sentence case...'); message = stopShouting(message); }
// Moderate if the user uses SwearWords. if (containsSwearwords(message)) { console.log('User is swearing. moderating...'); message = moderateSwearwords(message); }
return message; }
// Returns true if the string contains swearwords. function containsSwearwords(message) { return message !== badWordsFilter.clean(message); }
// Hide all swearwords. e.g: Crap => ****. function moderateSwearwords(message) { return badWordsFilter.clean(message); }
// Detect if the current message is shouting. i.e. there are too many Uppercase // characters or exclamation points. function isShouting(message) { return message.replace(/[^A-Z]/g, '').length > message.length / 2 || message.replace(/[^!]/g, '').length >= 3; }
// Correctly capitalize the string as a sentence (e.g. uppercase after dots) // and remove exclamation points. function stopShouting(message) { return capitalizeSentence(message.toLowerCase()).replace(/!+/g, '.'); }
Note: the bad-words package uses the badwords-list package's list of swear words which only contains around 400 of these. As you know the imagination of the user out there has no limit, so this is not an exhaustive list and you might want to extend the bad words dictionary.
Image Moderation
To moderate images we'll set up a blurOffensiveImages function that will be triggered every time a file is uploaded to Cloud Storage. We set this up by using the functions.cloud.storage().onChange(...) trigger rule. We'll check if the image contains violent or adult content using the Google Cloud Vision API. The Cloud Vision API has a feature that specifically allows to detect inappropriate content in images. Then if the image is inappropriate we'll blur the image:
To blur the image stored in Cloud Storage, we'll first download it locally on the Cloud Functions instance, blur the image with ImageMagick, which is installed by default on all instances, then re-upload the image to Cloud Storage:
