FileMaker introduced its JSON functions in version 16. They’ve changed how many developers work, both as a protocol for handling data interchange within FileMaker and working with external APIs. The JSON functions are for working with text, but what if we run into a situation where we need to work with actual JSON files? Does FileMaker provide the tools to read and write .json files as well as JSON text?

Looking for our article and example file on the JSON Parsing Function? Here it is: FileMaker JSON Functons.

FileMaker JSON: Reading Files

JSON files are really utf-8 text files, and we have a few different ways of handling them in FileMaker.

The Insert From URL script step can be used to insert .json files that are local to the system or downloaded from an external site or API. The way these files are inserted depends on the target that’s specified in the script step. For example, if a variable or text field is specified as the target, then Insert From URL will simply read the .json file and insert it as text. As text, we can then use the JSONGetElement function to parse the specific values.

FileMaker JSON Script

Sample script For inserting JSON as text

If a container field is specified as the target in the Insert From URL step, then the .json file is inserted into the container as an actual JSON file.

FileMaker JSON Container Field

JSON file inserted into a container as .json file

Since .json files are actually utf-8 encoded text files, we can read the text right from the container using the TextDecode function and specifying utf-8 as the encoding. However, it’s also possible these .json files are getting into the container field using a method that’s not Insert From URL, and the option of specifying the target is not available. Hence, the TextDecode function is useful in that scenario as well.

Using TexDecode Script Step

Insert JSON as a file and then read contents with TextDecode

Writing JSON Files from FileMaker

There are also scenarios where you may want to store or send your JSON from FileMaker as a .json file rather than as text. For example, some APIs recommend that you upload a .json file if the payload is very large instead of writing the JSON text inline as part of your request.

Let’s say we want to create the following JSON from FileMaker data and then turn it into a .json file.

The first thing we’ll do is to create the JSON with our JSONSetElement function like this.

If we’re using FileMaker client, we can set the JSON into a text field and then use Export Field Contents and specify the file name with the .json extension and then re-insert the file into our container: we’re all set. However, if we want this to run on the server or WebDirect, we need to handle this differently as Export Field Contents is unavailable on the server and is limited on WebDirect. As discussed earlier, .json files are just utf-8 text files, so we can use the TextEncode function to convert our JSON text to a file. When we do this, we get a file named utf-8.txt, but we need this to be a .json file with a specific name. To change the file name and extension, we can encode the file as Base64 and then decode it and specify the file name with the Base64Decode function’s optional second parameter. Here’s an example script showing how we can create the .json file with the name Example2.json.

Using Base64 with JSON files

Create .json file by encoding and then decoding as base64

We now have a properly encoded .json file ready for an API or for storing as a file.

Export JSON files from FileMaker

.json file created by a FileMaker script

Conclusions

In addition to the JSON text functions like JSONSetElement and JSONGetElement, FileMaker also provides us with the TextEncode and TextDecode functions. These, in combination with the JSON functions, allow us to not only read and write JSON as text but to write as .json files as well, extending the platform to cover whatever our JSON requirement may be.

Over the past year, we’ve been building some web apps in React that talk to our customers’ FileMaker servers and to Google’s Cloud Firestore database. As part of that, we’ve begun working with Google’s flavor of FaaS (Function as a Service), Firebase Cloud Functions. Cloud Functions, along with AWS Lamba and Microsoft’s Azure Functions, are part of a growing trend towards serverless architecture. The commonly cited advantages to utilizing Cloud Functions over server-based functions are typically:

  • Zero maintenance: No servers to spin up, no infrastructure to deploy
  • Scalable: Google will handle scaling up automatically
  • Pay as you go: You will be billed only when the function runs, rather than having to maintain a server at all times

In addition, you’ll often hear that Cloud Functions have the advantage of being “event-driven,” meaning they can subscribe to events across a suite of cloud services and trigger based on those changes. This is true, but it’s not unique to Cloud Functions. One web app we’re building here, Datapage, relies on server-based, event-triggered functions to send updates to a FileMaker server based on changes in Firestore, and those functions are working just fine.

If these were the only advantages of Cloud Functions, there isn’t much of a case for us to use them. We already operate and maintain a server for Datapage, and we aren’t receiving so many peak requests (yet) that we’d be in danger of overwhelming that server. So why have we begun a foray into FaaS? There are subtle, less discussed advantages to Cloud Functions that were central to our decision to give them a try.

Firebase Cloud Functions

Less Configuration

Our first candidate problem for Cloud Functions was making regular backups of our production Cloud Firestore database. Google provides functions to export all of the database’s collections to a Cloud Storage bucket, insuring us against overwriting production data. We could handle this with the Google Cloud SDK and a cron job on our server, but that would require configuring credentials, environment variables, and maintaining the job.

At SeedCode, we aim to have new servers up and running within 10 minutes should our currently running production server encounter an unresolvable issue. If we were maintaining a cron job using the Google Cloud SDK, we’d have to install, initialize, configure, and authorize the SDK every time we spin up a new production server. This would significantly increase the complexity of our server initialization process, which we aim to keep as simple as possible. The last time you want to be debugging your init script is during a server outage.

Instead, we used a Cloud Function for this job. We have only one configuration step: our default GCP (Google Cloud Projects) user needs the right role to access our backup bucket. After that, we can deploy a function that utilizes Cloud Scheduler to run once every 24 hours, exporting our collections to Cloud Storage. If you weren’t counting, that’s four separate cloud services: Firestore, Storage, Functions, and Scheduler, all working together in harmony with a single configuration step which we will only have to make once.

Fewer Links in the Chain

We had another excellent candidate for Cloud Functions arise shortly after we began utilizing them for automated backups. As part of the in-app notifications we built for Datapage, we gave users the ability to mark subsets of their unread notifications as read. This is a potentially expensive operation: it requires first querying for unread notifications within the specified timeframe, then updating all of them with a new status. We could do this client-side with Firestore, but what if a user has hundreds or even thousands of unread notifications?

FileMaker and Cloud Functions

Unread notifications in Datapage.

Instead of handling this client-side, we can do it in a Cloud Function. The user’s device won’t have to send out dozens of batched updates. The client will only send a single request to the callable Cloud Function, and the function will handle the rest, reading and writing within Google’s platform, with its superior networking. Since we are designing this app with plans to take it mobile in the future, we’re doing ourselves a favor by limiting expensive operations client-side.

Additionally, by not executing this function on our server, we are limiting links in the chain. Our database and function will run together on Google’s infrastructure, so we won’t need to pull down a large query onto our server and then send our updates back up. Instead of client -> Google -> server -> Google, we’re going client -> Google. Are the performance gains real? Maybe, but that’s a discussion for another article. For now, it’s enough to appreciate the reduced complexity.

Logging Google Cloud Functions

Google Cloud Function Monitoring

Is your server providing you with a visualization of memory usage for each function?

A severely underrated feature of Cloud Functions is Google’s excellent suite of logging tools. Want to know if the execution time of your function is going up? Want to use full-text search to find a specific error by timeframe? Don’t want to cross-reference your version control with your error log and wish you could just see which function invocation happened on which version of your app? It’s all here, and it’s all accessible through a secure web UI. Third-party server logging libraries with parity with these features do exist, but Google’s requires zero setup.

Beyond Our Use Cases

There’s a lot more here that we simply don’t yet have a use for: leveraging Google’s machine learning APIs, handling exceptionally CPU-intensive tasks, and integrating with 3rd party APIs. However, we have found that there’s enough of an argument to deploy some of our functions into the cloud anyway. We have no plans to move to a completely serverless architecture. Still, some aspects of serverless make sense, even if you’re already going to the trouble to maintain a server yourself.

Postscript: Vendor Lock-In

Dependence on a single company’s products and infrastructure is a major commitment. But when it comes to Cloud Functions, Google has taken steps to mitigate that concern by introducing the Functions Framework. FF allows Cloud Functions to become portable using containers and a set of open-source libraries. Moving your functions from the cloud down onto a server you manage yourself is now possible and will go a long way towards making the jump to Google’s Cloud Functions more inviting.

We’ve updated our example scripts for notifying users that they may be creating a conflict. Earlier versions of the script were for DayBack Classic, and now we’ve added example code for the new DayBack for FileMaker 19. This customization is a great example of intercepting the event-editing script in FileMaker and reverting changes to an event if your script deems it necessary.

Find Conflicts FileMaker 19 Calendar

Watch a video of this in action and download the example code here:

Conflict detection for DayBack Classic

Conflict detection for the new DayBack for FileMaker 19 and higher

Enjoy!

“We cut our planning meeting time by 50%. We are now able to spend more time refining our backlog instead of mechanically completing our schedule. We are super excited about this!” – Jeremy Verrillio, Vice President, CIPS

Happy to share this integration of DayBack Calendar with the FileMaker solution Jeremy Verrillio created for Comprehensive Injury Prevention Solutions (CIPS). Our blog post at dayback.com describes the strategies employed, including syncing to Google Calendar to their availability in Calendly is always up to date.

FileMaker Calendar Sync

Really proud of the work Jason, Becca, and Ann put into creating such an elegant solution.

Case Study & Video at dayback.com/blog

If your web viewer code calls the FileMaker.PerformScript function when loading, it’s possible that it hasn’t been injected into the web viewer yet, and your code will fail with a breaking error. How can we harden our opening routine to ensure this breaking error doesn’t occur and provide the best experience for the user?

Part 3 of 3 of our series on Web Viewer Tips for FileMaker 19.

In a previous post on Web Viewer Tips we discussed how the new version of DayBack uses a familiar JavaScript and API pattern and the benefits of doing this. In this post, we’ll suggest bringing another common JS pattern into your app: try-catch.

FileMaker 19 Web Viewer Calendar

DayBack gets FileMaker data (Events and To-Dos) as part of its loading routine.

“Can’t Find Variable: FileMaker”

The web viewer will initiate making calls to the FileMaker file via the FileMaker.PerformScript function as the web viewer loads, but since FileMaker is also injecting the FileMaker.PerformScript function into the web viewer as it loads, it’s possible that the function hasn’t been injected before your code calls it. This is part of the asynchronous nature of JavaScript, sometimes the FileMaker.PerformScript function is there in time, and sometimes it isn’t. When it isn’t there, the error “Can’t find variable: FileMaker” is thrown. This is a breaking error, and your JavaScript will stop executing on the offending line.

In most situations, we can trap for this kind of error by testing for it in a simple If statement, something like:

However, this does not help and you’ll still get the “Can’t find variable: FileMaker” error even when referenced in the if statement and the code breaks.

Using Try-Catch Blocks

Using Try-Catch blocks is a more robust way of trapping for errors and exceptions in JavaScript. All code run within the Try block will complete even if there is a potentially fatal error. Instead of the error being reported at the browser level, it will be sent to the Catch block as its argument. Here’s an example where the alert function is misspelled.

If the “alret” function is called outside of the try block, then this error would break our code and halt its execution at the line. By wrapping it in the try block, the code will continue safely and pass the message to the console log.

Since we know the FileMaker.PerformScript function will eventually be injected into the web viewer, we’re going to use the try-catch blocks to simply retry the routine a few times so that the injection can catch up with our code. This is the function we came up with, and even though it’s only on web viewer load that we’re worried about the failure, for consistency’s sake, we use it for all our calls to the FileMaker file.

As you can see, the function will try to call FileMaker.PerformScript again up to 5 times. We don’t want to get stuck in an infinite loop, so providing a maximum number of retries is certainly a best practice, although I’ve never seen more than 1 retry be required for the FileMaker.PerformScript to be there and for the try block run without passing the error to the catch block. At the worst, users may experience a half-second delay during the loading routine instead of simply failing and needing to manually reload the web viewer.

Conclusions

The asynchronous nature of JavaScript can create timing issues. Although this error may seem particular to FileMaker and its web viewer, similar issues have existed for web developers for years:  so have the solutions they’ve come up with to handle them. JavaScript may feel new to many of us in the FileMaker world, but we can take comfort that there’s a wealth of experience out there that we can adapt to our solutions. If you encounter similar JavaScript issues with your web viewer apps, even though the exact terminology may be different from what we’re now doing in FileMaker, someone has likely solved the issue. Some googling and searching through Stack Overflow will likely provide the solution.

What are the benefits of building a hosted app for the web viewer in FileMaker? And how can we make this approach work in WebDirect where it looks like FileMaker only supports data URLs?

In a previous article, we looked at writing JavaScript for FileMaker 19 that matches a typical  JS pattern of making asynchronous calls to a data source and handling the results. One of the important takeaways of writing JavaScript this way is that we can now maintain a single code base for our DayBack Calendar app across all platforms. In this article, we’re going to look at taking this idea a step further. In addition to being able to maintain a single codebase, we can host that codebase on a single server and have the FileMaker web viewer simply reference the server’s URL.

Hosting Web Viewer Apps in FileMaker 19

DayBack as a hosted app in FileMaker Pro client

This is a pattern that we’ve become very familiar with: it’s how DayBack works in Salesforce using Salesforce’s Canvas toolkit.

FileMaker 19 Best Practices

DayBack as a hosted app in Salesforce Lightning

Why Host your FileMaker Web Viewer Add-On?

For DayBack Calendar, which works with different platforms both embedded, like the FileMaker web viewer, and in the browser, hosting offers us a single code base and a single host for us to maintain. If we had to maintain different repositories and servers for the different platforms, that would exponentially more complex with each new calendar source we added–both in terms of architecture and in terms of making decisions on which version gets which features deployed. However, there are other benefits beyond working with multiple platforms that FileMaker developers should consider.

All Users are on the Same Version

Having multiple versions of the same product out in production can make things extra difficult for your support team. Even though there are some scripts and schema to maintain on the FileMaker side, by moving most of the logic into the single JavaScript build we can cut down chasing if an issue a customer is facing is based on a versioning issue or not. New features and bug fixes get deployed simultaneously and while not all features can be used on every platform, the default becomes that all users get the new features.

We worked very hard to develop scripted, in-app updates for the previous version of DayBack, which was not hosted. And even though most updates were installed with a single click, users still didn’t always know to update. This meant that we’d occasionally hear from folks using a version of DayBack that was over a year old, having missed out on a ton of new features and bug fixes. And while our in-app update scripts (alongside tools like Otto) work to minimize the amount of work it takes to get a file back up to date, all these users wished DayBack had just been updated for them. That’s what hosting the file accomplishes.

Connecting to Other Apps

DayBack Calendar can use events from multiple platforms within a FileMaker web viewer, including Google Calendars, Office 365 Calendars, BaseCamp, and Google Sheets. This is difficult to do if the web viewer is based on a data URL since most APIs need to work with a hosted domain for authentication. It would be technically possible to handle this on the FileMaker side with its native cURL functionality, but this would be a lot of FileMaker specific code vs. being able to take advantage of our hosted app’s existing ability to authenticate and work with these APIs. By referencing our hosted app, all these other non-FileMaker sources just work inside FileMaker.

How It Works: Setting Up the Web Viewer

Hosted apps work in FileMaker Pro client’s web viewer without issue. All we need to do to get them to work is to set up our web viewer to reference our server with a few parameters to tell the app in which context it’s being accessed.

However, if we try this in WebDirect, the app will load, but the FileMaker.PerformScript function is not injected into the web viewer code so we have no way of pulling or pushing info between the FileMaker file and the web viewer. In WebDirect the web viewer reference must have the data:text/html prefix in order for the FileMaker.PerformScript function to be injected. In order to get this to work in WebDirect, a data URL is used for the actual page, so the FileMaker.PerformScript function is injected properly, and then include an iFrame that references the hosted app within the data URL. Here’s an example:

Using the iFrame in FileMaker 19’s Web Viewer: Sending Data In and Out

At this point, the iFrame is loading DayBack from the server, but the FileMaker.PerformScript function isn’t injected into the iFrame and can only be referenced in the scope of the data URL. iFrames are designed this way intentionally as the parent window and iFrame are typically from other domains and can introduce cross-site attacks. However, modern browsers do provide a way to safely communicate with iFrames via JavaScript if the proper protocols are set up on both pages.

To send a message from an outer page to an iFrame, the JavaScript function the postMessage window method is used. From the parent page, the iFrame element can be identified by its ID and a message sent.

In the hosted app in the iFrame, an event listener can be added when the page loads to listen for the message and then run the appropriate function, in this case, postToDayBack.

To send a message from the hosted code in the iFrame to the parent window a similar process is set-up in reverse. Here the domain will be set as a wildcard so we can post the message to a data URL.

Then a listener is added to the data URL which can then receive the message from the iFrame and use the FileMaker.PerformScript function to relay that to the file.

The reason we test for the dbk property in the message is that the FileMaker script step Perform JavaScript in Web Viewer also uses the postMessage method and this will trigger the listener that’s been we’ve added to the data URL ensuring that we only messages coming from the DayBack iFrame will actually be processed and execute the FileMaker.PerformScript function.

Essentially, the data URL serves as a relay between FileMaker and DayBack since they can’t communicate with each other directly in WebDirect. The idea to do this in FileMaker was mostly inspired by our experience with bringing DayBack to the Salesforce platform as the Canvas toolkit we used there actually provides developers with ready-made tools for doing this kind of communication. So, again we have similar patterns being used across different platforms which helps simplify DayBack’s code significantly. And we have a familiar pattern for getting DayBack working as a hosted app in WebDirect.

Web viewer add-on FileMaker 19 Web Direct

DayBack as a hosted app in FileMaker WebDirect

Conclusions

A hosted app can dramatically reduce the amount of effort required to deploy and maintain code being used by a FileMaker web viewer app. It also provides a centralized way of allowing your app to communicate with other APIs without having to reinvent the wheel with FileMaker scripts.  Although there are a few extra steps to making things work in WebDirect, there’s actually not that much additional code needed. Using postMessage and message listeners is also an established JavaScript pattern with tons of examples and best practices available to get you started. In addition to these benefits, if you build your app this way, you’re probably much closer to taking your app beyond the FileMaker platform than you may realize.

Happy hosting!

Is this your DayBack story? You have users who are primarily using one calendar view for a long time, and another user updates an event.  In a busy scheduling environment–like a call center–how does everyone know that something has changed? Normally other users won’t see these changes until they change a filter, change their view, or click the calendar’s refresh button. This can slow folks down or poorly schedule a particular resource or timeslot thanks to out of date information.

Listening for Other Users’ Changes in DayBack for FileMaker

We recently had an opportunity to modify a customer’s DayBack for FileMaker 19 to automatically refresh any modified or deleted events for other users viewing the same data. Imagine – the front desk receives a call from a customer who needs to change an appointment to later in the day.  For staff in the back office, this is reflected in DayBack just seconds later with no user interaction required. This is a game-changer for our customer, and perhaps it will be for you as well!

We’ve uploaded a simplified version of what we created that you can paste into your own deployment of DayBack (example file and instructions follow below). Here’s a short video of this customization in action:

 

How It Works: Execute FileMaker Data API

When a user makes a change to an event, a field on that record captures the host’s current timestamp of when the modification was made.  By using the host timestamp, we can ensure that the modification timestamps are consistent, regardless of the time zone any individual user may be in. An on timer script will run for each user, and if it finds that there are events that have been modified, it will update the display of these events.

Read Full Article →

We’ve found react-redux-firebase (RRF) to be a big time saver when querying Firestore. Despite the name, the library has robust support for not just Firebase, but Firestore too, pulling in redux-firestore behind the scenes for Firestore operations. The documentation for RRF’s use of redux-firestore’s APIs is scant within the RRF docs themselves, but redux-firestore’s documentation is extensive and can be applied to usage of RRF.

Paginating Firestore Queries react-redux-firebase

 

We make use of paginated Firestore queries to infinitely scroll a particularly long list of items. Pagination in Firestore is accomplished through a set of query configuration options that use cursors to determine where to start or end the page of your query. Here’s what redux-firestore’s docs have to say about using a cursor clause to paginate a Firestore query: Read Full Article →

Should we trust the script execution order in FileMaker? Or should we write web viewer code that doesn’t care?

In FileMaker 19, developers can now use JavaScript in their web viewer apps to call FileMaker scripts directly. And when those FileMaker scripts complete, they can, in turn, call JavaScript in the web viewer to pass results back. While it’s tempting to assume these two things will happen right after each other, it can be more useful to write JavaScript that doesn’t expect FileMaker to hand its results right back.

This article describes writing “asynchronous JavaScript” that’s much more forgiving of different script behaviors in FileMaker and, at the same time, is much more like the JavaScript you’d write when connecting to sources other than FileMaker. As a result, JavaScript developers on your team may find this asynchronous pattern more recognizable than FileMaker-specific methods.

FileMaker 19 JavaScript

Background: JavaScript in the FileMaker Web Viewer

There’s a lot of excitement over the new JavaScript enhancements introduced in FileMaker 19 that allow bi-directional communication between FileMaker and the web viewer. This means that developers can now write JavaScript apps for FileMaker that follow the typical pattern of sending a request to a data source, receiving a response asynchronously, and then updating the view without having to do any kind of refresh of the web viewer. Experienced JavaScript developers can now approach writing a JavaScript app for FileMaker without learning all the workarounds required to achieve something similar before these enhancements were introduced in version 19.

Technically, it has been possible to do this kind of communication since FileMaker 13, but it involved many complex processes, often using FileMaker’s temp directory. Doing this also relied on the FMPURL protocol, which couldn’t talk to more than one FileMaker version at a time. Worse still, these methods wouldn’t work in Web Direct since neither the temp directory nor the FMPURL protocol is supported there. What you would typically end up with is a JavaScript architecture that was unique to supporting FileMaker. Read Full Article →

DOWNLOAD
DayBack Calendar
DayBack's 30-day trial is unlocked so you can customize it and integrate it with your files.
Download DayBack and we'll send you a couple short emails with tips on how to modify it and use some of the coolest features.
Thank you! Please download: DayBack Calendar
Need More?
SeedCode tips & example files in your inbox
Need More?
SeedCode tips & example files in your inbox
Want More?
Be the first to see articles and tips like these
DOWNLOAD
TimeZync
Download TimeZync and we'll send you a couple short emails with tips syncing your FileMaker Go files.
Thank you! Please download: TimeZync
Want More?
Be the first to see articles and tips like these