AI, and Assistants

As an undergraduate, I did fairly extensive work in early text-based virtual communities, particularly the way in which people interacted and communicated in text-based virtual realities. The research may color my vision of Duplex and other technologies that make it harder for people to know who they are speaking with.

This makes the argument that we have the “right” to know that we’re speaking to a ‘bot harder to accept for me. Unless you know the person you’re speaking with how can you really tell who are you communicating with? How do you know that the knowledge they claim is factually correct and that they even possess it, to begin with?

The issue, as I see it

I think that the problem began with the Duplex Demo at Google I/O this year. It wasn’t a complete app demo and it was a very limited show of what both the assistant and a new technology called Duplex.

People jumped the gun and started accusing Google, and Amazon before them, to be trying to create machines that fool humans or that not enough care has been put into the ethics and morals of such interactions.

Enhancements to Google Assistant

Google assistant for Android and, in a much more limited capacity, iOS will be enhanced with interesting features:

New voices
Google’s Wavenet technology is bringing improvements to computer voices in general and the Assistant voices in particular, creating system voices that sound more natural and human-like than before. Assistant has added six new voices using this technology.
Also as part of Wavenet, John Legend’s voice will be coming for some context using Assistant later this year.
Continued Conversation
You no longer have to trigger Assistant with hot words like ‘Hey Google’ and ‘OK Google’ before every sentence, allowing you to have a more natural conversation. Assistant will be able to distinguish between when you are talking to it or to another person.
Multiple Actions
Multiple Actions allows Google’s Assistant to perform simultaneous actions in the same voice command, using coordination reduction to figure out exactly what the user means even with longer commands.
Pretty Please
Google Assistant is adding skills that compliment and provide positive reinforcement to users (especially kids) when they converse with the Assistant using polite words like Please and Thank You.

I think it’s the first point that makes people nervous when it comes to the Assistant. If the voices on your devices become harder to distinguish from a human or if we can have long conversations where the agent/assistant can interpret what we’re saying without us having to take specific actions, it all leads to apprehension and fear.

An aspect of this “fear the intelligent machine” is that, in giving agency and independent action capabilities to a piece of software, we take away that agency from ourselves… We don’t make reservations ourselves, we let the machine make them for us… What will we give up next?

But it pays to remember that the assistant will not do anything on its own. Even if you somehow used the assistant to perform a highly controversial action, there was still human agency in the action, there was no self-directed AI that actually pulled the trigger.

That’s where the real concerns should be… How do we ensure human control over the actions of our AI agents?

Duplex

As described in the blog post announcing the technology: Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone:

The technology is directed towards completing specific tasks, such as scheduling certain types of appointments. For such tasks, the system makes the conversational experience as natural as possible, allowing people to speak normally, like they would to another person, without having to adapt to a machine.

One of the key research insights was to constrain Duplex to closed domains, which are narrow enough to explore extensively. Duplex can only carry out natural conversations after being deeply trained in such domains. It cannot carry out general conversations.

So we see in these paragraph describing the technology we can see a few things:

  • It’s task-specific and trained in specific domains
  • It is not an open-ended conversational agent. It won’t initiate communication on its own
  • Using a more human voice makes it easier to interact without having to face the notion you’re dealing with a machine, and potential rejection that talking to a machine may cause

Further down the article there is a statement that I find interesting and that prompts a series of questions:

The system has a self-monitoring capability, which allows it to recognize the tasks it cannot complete autonomously (like scheduling an unusually complex appointment). In these cases, it signals to a human operator, who can complete the task.

  • Are these operators always available when Duplex is making a call?
  • Are these operators similar to the drivers in a “self-driving” car?
  • How do hand-offs to a human operator happen? How does the machine evaluate a call as ‘unusually complex’?
  • How has the human at the other end of the conversation reacted when Duplex handed the call to a human? Did the call complete as expected?

It is also important to remember that, like Assistant, Duplex can only be triggered by human interaction. You tell Assistant that you want to make a reservation and Assistant uses Duplex to make the reservation, it won’t happen without a human in the loop to initiate the AI action.

Ethical And Moral Considerations

As much as I’m for technology like Google Assistant and Duplex, there’s a lot to be worried about, but I also think that right now we’re worried about the wrong things and we’re only considering potential wrongs that the technologies may have.

DeepMind created a research group around Ethics & Society. One area of research for this group is Privacy, transparency and fairness. I think this is where the biggest short-term value for both AI researchers and for the communities that they seek to serve. If we can show that the AI agents, both monitored and autonomous, can treat privacy and transparency in the same way humans can it’ll go a long way in calming some of the fears people have. It won’t settle them completely as people say they want to talk to ‘real people’ and ‘not to a machine’.

Another research area that I find intriguing is AI morality and values. I find it interesting because it asks one of the questions that I’ve always had about technology, not just AI: How can we program morals and ethics into a system when we, or the people who will actually control the AI, may not share or have those morals or have a different ethical viewpoint than we do? As our machines become smarter than we are, better networked than we are and achieve global reach, there will be a commensurately larger need for machines that can interpret and act upon our values (whatever they may be). Perhaps the most important question is what will happen when AIs with different morals, programmed by people with different moral, religious and ethical backgrounds enter into conflict with each other? With humans?

Will we have unleashed one or more Skynet?

Ethics and morals are hard to regulate and impossible to program. While it’s true that we can agree on some basic standards it’s also true that even those agreed upon standards are not universal.

Furthermore, whose morals, ethics, and biases should we feed into our AI tools? Are these the same for autonomous AI? What happens if the ethical subroutines in the machine are different than mine?

An interesting example of programming morals are Asimov’s three laws of Robotics, first appeared in Runaround (March 1942 issue of Astounding Stories of Super-Science). The three laws are:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm
  2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws

This is as close as we’ve managed to get to codifying the behavior or autonomous robots and artificial intelligence; but it’s also interesting to see that in many of Asimov’s stories, Robots circumvent the laws in one form or another because of its own actions or those of a human involved.

How bulletproof will the morals we that we’re allowed to input in our AI agents?

Deception is not exclusive to machines. In Identity and Deception in the Virtual Community, and Being Real, Judit Donath discusses identity and deception in the early virtual communities of the 1990s. Donath points out that:

“One can have…?” Who is this “one”? It is, of course, the embodied self, the body that is synonymous with identity, the body at the keyboard. The two worlds are not truly disjoint, but their relationship is murky and intricate. A single person can create multiple electronic identities, their only link their common progenitor, a link which is invisible in the virtual world. A man can create a female identity; a high-school student can claim to be an expert in virology. If the two worlds were truly unconnected, the result might be simply a wholly independent society, wherein a virtual identity would be taken on its own terms. Yet this is not the case. Other explorers in virtual space, males in the real world, develop relationships with the ostensible female, and are shocked to discover her “actual” gender. The virtual virologist’s pronouncements on AIDS breakthroughs and treatments are assumed to be backed by real-world knowledge. Do virtual personas inherit the qualities – and responsibilities – of their creators?

From Identity and Deception in the Virtual Community

AI doesn’t make the issue necessarily worse, some people have decided that it’s far worse when a machine does the deception upon human instructions and when we can’t tell the difference between human and machine. Is it just a difference of degrees that makes us doubt the machine any more than the person who sent us the spear phishing email to try to compromise the business we work for?

In the late 1990s and early 2000 scientists and people worried that we were becoming too familiar with technology to the point where we’ve started losing the ability to communicate with other humans but I wonder if this being a problem is actually a generational issue.

This is not to say we should be complacent either. The fact that we’re getting comfortable with the technology doesn’t mean we should become lax in enforcing and understanding a common set of best practices for interacting with AI and how AI behaves when interacting with us.

Kate Crawford presented an interesting lecture at the University of Washington’s Tech Policy Lab on the social and political implications of AI.

We have to ask ourselves if we’re bringing our biases into AI and machine learning. I don’t think is possible to take our biases into the AI world? Why is it that, as Crawford, most biases in AI are reflections of systemic biases present in our culture since before the last wave of AI came into the popular consciousness?

Who would benefit the most?

So far I’ve also fallen into the trap of just talking about the negative potential of AI and conversational agents.

The first group that comes to mind is those who rely on computer-assisted speech for day-to-day interactions.

Another group that would benefit from having a conversational agent are people who suffer from social anxiety disorders or who, like the hikikomori in Japan, choose to have little or no contact with the outside world.

How do we reconcile the needs of groups like this with our desire for privacy and the need we all have to communicate?

Do the good aspects overwhelm the bad? How do we use machine learning, deep learning, and other AI applications to ameliorate the underlying systemic biases that caused the technological biases we are becoming aware of?

Conclusion: Bringing It All Together

AI is an interesting and dangerous topic. I think, for the purpose of this final discussion, we can group it in two areas: Human/AI interaction and fully autonomous AI.

Human/AI interaction is the most recent and, apparently, the most controversial. We’re afraid that the machine will “trick” us when talking to us rather than see the conversation as a mutually beneficial exchange it is.

There are at least three issues that we need to address when talking about human/AI interaction:

  • How do we set ethical, moral, and legal boundaries for AI?
  • How do you build trust from humans towards AI?
  • As conversational UI and AI developers, how do we address users’ concerns?

Autonomous AI presents a different set of challenges that are a larger degree of biases both cultural and against AI. Autonomous, self-sufficient AI, is too wide a topic to include here, but it’s important to keep in mind.

How we deal with these issues and build comfort with the technologies will have huge implications for society and the individual.

Links and Resources

Reproducible Configs

A lot of the projects that I work with use a fairly standard setup of tools and scripts. Rather than reinstall them every time and have to fix paths and install dependencies outside the Node ecosystem I’ve created a script that will perform the following actions:

  1. Check if package.json exists. If it doesn’t then create one with default values
  2. Install packages explaining what it’s doing when installing each group
  3. Run an additional script that will create configuration files
  4. Remove the .git directory and recreate it for the current directory

This way I can initialize a new project using 2 files that will create a reproducible starting point yet retaining a degree of flexibility.

That’s why we don’t force an ESLint configuration. If I’m writing React I’ll use Airbnb preset as it is geared towards working with React. For most other projects I’ll use Google’s preset; it is what I became comfortable with when developing PWA content and I think it’s still the one that makes most sense.

There are two directories and a file that I’ve included to use for my type of projects. I create a src directory with the structure of files that I will use in the Gulpfile.

The first thing we do is check if there is a package.json file in the directory we’re running the script in. If there is one we exit. I’d rather not overwrite the file that is there.

If there isn’t one we initialize a default package.json file that we’ll install packages to.

#!/usr/bin/env bash

echo "Checking if package.json exists"
if [ -f package.json ]
then
echo "package.json exists."
exit 1
else
echo "package.json doesn't exists... creating"
npm init --yes
fi

The rest of this script installs different sets of packages. I will only highlight any specifics that I think are important.

echo "installing NPM packages "

echo "Installing Gulp 4.0 and load-plugins"
npm i -D [email protected] gulp-load-plugins

The first big change I’ve made is to fully embrace Gulp 4.0. I will modify the file to take into account the new methods it uses.

echo "Installing PostCSS and related plugins"
npm i -D postcss autoprefixer cssnano gulp-postcss

echo "Installing Babel Core and Preset Env"
npm i -D @babel/core @babel/preset-env

echo "Installing Browser Sync"
npm i -D browser-sync connect-history-api-fallback

echo "Installing ESLint"
npm i -D eslint gulp-eslint

It’s important to notice that we’re only installing eslint. We’ll handle the preset installation later

echo "Installing SASS and related plugins"
npm i -D gulp-sass gulp-sourcemaps

echo "Installing critical and uncss"
npm i -D critical uncss

echo "Installing Remarkable and related plugins.  I use Remarkable and associated plugins to generate production-ready HTML"
npm i -D remarkable gulp-remarkable gulp-wrap

I use Markdown as my authoring tool. I then use Remarkable and gulp-remarkable to convert it to HTML and gulp-wrap to put the content inside a template that has links to the CSS and Javascript that I use for particular projects. All we need to do to use different assets is to change the links in the template.

echo "Installing miscelaneous plugins"
npm i -D  gulp-concat gulp-debug gulp-exec gulp-run gulp-size

echo "The following plugins are optional. "

echo "Installs Google Cloud utilities to upload content to google cloud storage. Including gulp-gzip for compression.  Separating them to make sure this is still the best option"
npm i -D gulp-gcloud-publish gulp-gzip

The original project created PDF assets that were the uploaded to a Google Cloud storage bucket. I haven’t decided if I want to continue using GCloud Storage that way or if I want to move to Firebase hosting instead.

echo "Utility Opentype adds classes for using Opentype font options"
npm i -D utility-opentype

echo "Cheerio provides the means to query HTML and XML structures from a Gulp file."
npm i -D cheerio

echo "Axe is an accessibility evaluation tool."
npm i -D gulp-axe-webdriver

echo "Evaluates your node_modules file for vulnerabilities.  This is the tool that NPM runs when using npm audit."
npm i -D gulp-snyk snyk

echo "External Dependecies Required"
echo "SASSDoc and SCSS Lint depend on external Ruby dependencies."
npm i -D sassdoc scss-lint gulp-scss-lint gulp-scss-lint-stylish

echo "Running configuration script"
sh configure.sh

Rather than make the script longer than it needs to be I call a second script to do configuration. This script is more complex and uses some Bash tricks.

#!/usr/bin/env bash

# Configure ESLint
echo "ESLint configuration"
echo "Checking if .eslintrc.json exists.  We're assuming it was written as a Javascript file"
if [ -f .eslintrc.js ]
then
echo ".eslintrc.json exists."
exit 1
else
echo ".eslintrc.json not found. Creating... "
echo "Please answer the questions below:"
npx eslint --init
fi

If the .eslintrc.js file exists then we tell the user and exit. If it doesn’t exist then we run npx eslint --init to configure eslint based on our preferences and the needs of the project.

# Creating .babelrc file
if [ -f .babelrc ]
then
echo ".babelrc exists"
exit 1
else
echo "creating .babelrc"
fi

touch .babelrc
cat > .babelrc <<- "EOF"
{
"presets": [
["env", {
"targets": {
"browsers": ["last 2 versions", "safari >= 7"]
}
}]
]
}
EOF

Configuring babel is slightly different. We test like we did with ESLint. When it doesn’t then we create the file using the touch command and then append the content of the file.

# Create and populate .editorconfig
if [ -f .editorconfig ]
then
echo ".editorconfig exists"
exit 1
else
echo ".editorconfig doesn't exist"
echo "creating..."
fi

touch .editorconfig
cat > .editorconfig <<- "EOF"
# http://editorconfig.org
root = true

[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

# Use 4 spaces for the Python files
[*.py]
indent_size = 4
max_line_length = 80

# The JSON files contain newlines inconsistently
[*.json]
insert_final_newline = ignore

# Minified JavaScript files shouldn't be changed
[**.min.js]
indent_style = ignore
insert_final_newline = ignore

# Makefiles always use tabs for indentation
[Makefile]
indent_style = tab

# Batch files use tabs for indentation
[*.bat]
indent_style = tab

[*.md]
trim_trailing_whitespace = false
EOF

editorconfig provides a standard way to tell your editor how to behave. The (defacto) standard is supported by most modern text editors.

The file we’re adding to the project is a basic default that we can use for most projects. Check the website for additional information.

# Git configuration has to be the last step to make sure
# that we get all the files we want are committed to the repo
echo "Reseting GIT configuration.Because I downloaded this as a Git Repo clone I want to remove it and start over"
echo "If you're working with an existing project this may loose your project's history"
echo "Removing .git directory"
rm -rf .git
echo "creating empty git repository"
git init
echo "creating .gitignore"
touch .gitignore
echo "adding node_modules and shell files to .gitignore"
#
echo "node_modules" >> .gitignore
echo "install.sh" >> .gitignore
echo "configure.sh" >> .gitignore
#
echo "adding files to repository"
git add .
echo "commit initial files"
git commit -am "initial commit"

Working with Git in this setup is the last step in the process. It is important to note that if you’re working on an existing project this will blow up your project’s local history.

The first step of the process is to remove the existing .git directory. I choose to do it tto make sure that the next steps are done on what, for Git, is a clean repository.

Next we create a .gitignore file and add the files we don’t want to upload to Git. In this case I’ve chosen not to upload the shell (install.sh and configure.sh files and the node_modules directory.

Then we add the files to the repository and commit them. All that is left is to push them to your Git or Github repo.

Asynchronous Javascript: SetTimeout and SetInterval

SetTimeout and SetInterval provide ways to schedule tasks to run at a future point in time.

setTimeout allows you to schedule a task after a given interval.

setInterval lets you run a task periodically with a given interval between runs.

SetTimeout

setTimeout takes two parameters:

A string representing code to run and a number representing the time interval in milliseconds to wait before executing the code.

In the following example, the browser will wait two seconds before executing the anonymous function and presenting the alert message.

let myGreeting = setTimeout(function() {
  alert('Hello, Mr. Universe!');
}, 2000)

We’re not required to write anonymous functions. The second version of the example uses sayHi as the name of the function. The rest of the code remains unchanged.

let myGreeting = setTimeout(function sayHi() {
  alert('Hello, Mr. Universe!');
}, 2000)

The code is rather messy. We can clean up the setTimeout call by taking the function outside the setTimeout call. The next iteration of our code defines sayHi first and then references the function by calling sayHi without parenthesis as the first parameter of setTimeout.

function sayHi() {
  alert('Hello Mr. Universe!');
}

let myGreeting = setTimeout(sayHi, 2000);

The last step in the demo is to pass parameters to the function we want to use in setTimeout.

This gets a little tricky.

First, we configure the function to add the parameter and use it in the body of the function.

When we call setTimeout we pass the values for the function parameters as the third (and subsequent if there is more than one) parameters.

function sayHi(who) {
  alert('Hello ' + who + '!');
}

let myGreeting = setTimeout(sayHi, 2000, 'Mr. Universe');

All versions of the function will produce the same result… but they show different ways we can use setTimeout and the flexibility we have in writing the code.

ClearTimeout

This is less of an issue with setTimeout as it is with setInterval (discussed in later sections) but there may still be situations where you want to abort the execution of code inside a setTimeout call. For example, let’s say that we set the timeout for a very expensive task.

function runExpensiveTask() {
  alert('Expensive Task has been completed!');
}

let myTaskRunner = setTimeout(runExpensiveTask, 30000);

And we want to stop it because we want to do something else on the page. To do it we call clearTimeout with the id of we assigned to setTimeout when we created it.

let forgetIt = clearTimeout(myTaskRunner);

clearTimeout() and its cousin clearInterval() use the same list of entries to clear from. This means that you can use either method to remove a setTimeout or setInterval. For consistency, I use clearTimeout to clear setTimeout() entries and clearInterval to clear setInterval() entries.

setInterval

setTimeout works perfectly when we need to run the code once after a set period of time. But what happens when we need to run the code every x milliseconds?

That’s where setInterval comes in. When we use this command, the code we attach to it will run every time the interval completes.

The example below creates a new date object and logs it to console. We then attach it to setInterval and execute it once every second. This will create the effect of a running clock that updates once per second.

function countTime() {
    let date = new Date();
    let time = date.toLocaleTimeString();
    document.getElementById('demo').innerHTML = time;
}

const createClock = setInterval(countTime, 1000);

clearInterval

With repetitive tasks like our clock example, we definitely want a way to stop the activity… otherwise, we may end up getting errors when the browser can’t complete any further versions of the task.

stopTime() clears the interval with the ID we created.

The example goes further and creates a button and attaches the stopTime function to the button’s click event so when we click the button the interval will stop.

function stopTime() {
    clearInterval(createClock);
}

let myButton = document.getElementById('stopButton');
myButton.addEventListener('click', stopTime);

Things to keep in mind

There are a few things to keep in mind when working with setTimeout and setInterval.

Recursive Timeout

There is another way we can use setTimeout: Use it recursively to call the same code repeatedly instead of using setInterval.

Compare the first example using a recursive setTimeout to run the code ever 1000 milliseconds.

let i = 1;

setTimeout(function run() {
  console.log(i);
  setTimeout(run, 100);
}, 100);

The second example uses setInterval to accomplish the same effect of running the code every 100 milliseconds.

let i = 1;

setInterval(function run() {
  console.log(i);
}, 100);

The difference between the two versions of the code is a subtle one.

Recursive setTimeout guarantees a delay between the executions; the code will run and then wait 100 milliseconds before it runs again. The 100 milliseconds will happen regardless of how long the code takes to run.

Using recursive setTimeout guarantees the interval will be the same between executions. Image taken from https://javascript.info/settimeout-setinterval.

The example using setInterval does things differently. The interval we choose for setInterval includes the code we want to run in its execution. Let’s say that the code takes 40 milliseconds to run, then the interval ends up being only 60 milliseconds.

The interval we set for setInterval includes our own code execution. Image taken from https://javascript.info/settimeout-setinterval.

Immediate Timeout

Using 0 as the value for setTimeout schedules the execution of func as soon as possible but only after the current code is complete.

For instance, the code below outputs “Hello”, then immediately “World”:

setTimeout(function() {
  alert('Mr. Universe')
}, 0);

alert('hello');

When would you use them?

setTimeout and setInterval are useful when you need to schedule code execution.

Use setTimeout if you want to execute the code once after a given time elapses. Pay attention to the gotchas for using setTimeout and consider them additional alternatives

Use setInterval if you need the code to execute repeatedly at given intervals.

Asynchronous Javascript: Introduction and Synchronous Code

When we write Javascript we normally do it synchronously, every instruction is executed one at a time in the order they appear in the script. The script will finish once the last instruction is executed.

In the example below, we log three items to the console.

console.log('1');
console.log('2');
console.log('3');

We will always get the same result: 1 2 3.

This works great for small pieces of code or scripts that don’t produce output on the browser. But sooner rather than later you will want to work on larger scripts and you will find one of the first issues with Javascript: It blocks rendering.

Let’s take the following HTML document.

<!DOCTYPE html>
<html lang='en'>
  <head>
    <meta charset='UTF-8'>
    <meta name='viewport' content='width=device-width, initial-scale=1.0'>
    <title>Document</title>
    <script src='myscript.js'></script>
  </head>
  <body>

  </body>
</html>

And the following content in myscript.js:

// Capture the body tag as the third child of the document (zero based)
const content = document.documentElement.childNodes[2];
// Create an h2 element and assign it to the header constiable
const header = document.createElement('h2');
// Add content to the header
header.innerHTML = 'This was inserted';
// Assign it as the first child of the body element
content.insertAdjacentElement('afterbegin', header);

The browser will load the page and when it finds the script it will load it and process it. Because the browser can not know ahead of time if the script will insert, remove or change content on the page it will pause rendering until the script is fully parsed and any changes to the page like inserting or removing nodes are done. The script can have any number of functions and interact with databases and other resources on your page and application.

This will happen for every script that you add to the page; they will each load completely before moving onto the next. Working with smaller scripts may be fine but imagine if you’re loading jQuery from a Content Delivery Network and then loading our script. Our HTML will look like this:

<!DOCTYPE html>
<html lang='en'>
  <head>
    <meta charset='UTF-8'>
    <meta name='viewport' content='width=device-width, initial-scale=1.0'>
    <title>Document</title>
    <script
      src='https://code.jquery.com/jquery-3.3.1.min.js'></script>
    <script src='myscript.js'></script>
  </head>
  <body>
    <!-- content goes here -->
  </body>
</html>

Even with no content, this page will take longer to load as it now has to download jQuery before it can continue processing the page and its content, which may include other scripts.

Things will get slower when the code we write does heavy processing like database access or image manipulation.

The figure below gives you an example of what the execution order or stack looks like for a synchronous blocking language like Javascript.

Before we jump into asynchronous (async) code we’ll look at two ways of writing synchronous code: Callbacks and try/catch blocks.

Callbacks

Callbacks are functions that are passed as parameters to other functions.

An example of a callback is the second parameter of an event listener. In the example below, clicking the button with id of myButton will trigger the function and produce the alert for the user.

myButton.addEventListener('click', function() {
  alert('You Clicked THE Button');
})

Another example is when we use forEach to loop through the items in an Array. In this case forEach will loop through the array

const gods = ['Apollo', 'Artemis', 'Ares', 'Zeus'];

gods.forEach(function (eachName, index){
  console.log(index + 1 + '. ' + eachName);
});

The two examples work because Javascript allows functions as parameters in other functions.

When we pass a callback function as an argument to another function, we are only passing the function definition. We are not executing the function in the parameter. The containing function can execute the callback anytime.

Note that the callback function is not executed immediately. It is “called back” (hence the name) somewhere inside the containing function’s body synchronously.

Working with ‘this’

When the callback function uses the this object, we have to modify how we execute the callback function to preserve the this object context. Or else the this object will either point to the global window object (in the browser) if callback was passed to a global function. Or it will point to the object of the containing method.

const agentData = {
  id: 007,
  fullName: 'Not Set',
  // setUserName is a method on the agentData object
  setUserName: function (firstName, lastName)  {
    this.fullName = firstName + ' ' + lastName;
  }
}

function getUserInput(firstName, lastName, callback)  {
  if ((firstName.length > 3) && (lastName.length > 3)) {
    callback (firstName, lastName);
  } else {
    console.log('data could not be saved.');
  }
}

When we create a new agent something unexpected happens. agentData.fullName returns the default value of Not Set but when we check for fullName in the window object (window.fullName) we get the name we expected.

getUserInput ('James', 'Bond', agentData.setUserName);
console.log (agentData.fullName);
console.log (window.fullName);

The problem has to do with this.

In most cases, the value of this is determined by how a function is called. It can’t be set by assignment during execution, and it may be different each time the function is called.

When we call a function directly, like we call getUserInput, the context is the root object (window in the case of the browser)

When a function is called as a method of an object, its this is set to the object the method is called on.

So how do we solve this problem?

Call and Apply

We can fix the problem with this using call or apply. These methods (available to all functions) are used to set the this object inside the function and to pass arguments to the functions.

Call takes the value to be used as the this object inside the function as the first parameter, and the remaining arguments to be passed to the function are passed as a comma-separated list.

For call and apply to work we add an extra parameter that designates what object we want to use to represent this, in this case, we call it callbackObj.

The biggest difference is that our new version of getUserInput uses the call method of the callback function with four parameters, including the new callbackObj.

const agentData = {
  id: 007,
  fullName: 'Not Set',
  // setUserName is a method on the agentData object
  setUserName: function (firstName, lastName)  {
    this.fullName = firstName + ' ' + lastName;
  }
}

function getUserInput(firstName, lastName, callback, callbackObj)  {
   if ((firstName.length > 3) && (lastName.length > 3)) {
    callback.call (callbackObj, firstName, lastName);
  } else {
    console.log('data could not be saved.');
  }
}

To verify that it works we add a new agent with our getUserInput function and, here is the key, we tell it that we want to use agentData as the object to represent this.

When we log the value of agentData.fullName we get the expected value. When we try to log the value of window.fullName we get undefined because this means agentData not the global window object.

getUserInput ('James', 'Bond', agentData.setUserName, agentData);

console.log (agentData.fullName); // James Bond
console.log (window.fullName); // undefined

The Apply function’s first parameter is also the value to be used as the this object inside the function, while the last parameter is an array of values to pass to the function.

The difference between apply and call is the signature of the method.

const agentData = {
  id: 007,
  fullName: 'Not Set',
  // setUserName is a method on the agentData object
  setUserName: function (firstName, lastName)  {
    this.fullName = firstName + ' ' + lastName;
  }
}

function getUserInput(firstName, lastName, callback, callbackObj)  {
   if ((firstName.length > 3) && (lastName.length > 3)) {
    callback.apply (callbackObj, [firstName, lastName]);
  } else {
    console.log('data could not be saved.');
  }
}

The signature for getUserInput doesn’t change when we use apply instead of call. The results are identical.

getUserInput ('James', 'Bond', agentData.setUserName, agentData);

console.log (agentData.fullName); // James Bond
console.log (window.fullName); // Undefined

Try/Catch blocks

The second way to write synchronous code is to create try/catch blocks.

The idea behind try/catch blocks is that we need to run instructions in sequence but we must also do something if any of the commands fail.

We will use the following JSON for the example.

// data from the server
let json = '{"id":"007", "firstName":"James", "lastName": "Bond"}';

In the try/catch block below, we want to make sure that the data parse succeeds and do something if there is an error.

In this example, we just log a message and the error to console. In more complex examples the catch error may attempt connecting to the database again or ask the user to enter the data again if it wasn’t complete or well formed.

try {
  // convert the text representation to JS object
  let user = JSON.parse(json);
  // Log the results to console
  console.log (user.id); // 007
  console.log (user.firstName);  // James
  console.log (user.lastName); // Bond
} catch(err) {
  console.log ('I\'m sorry Dave, I can\'t do that ');
  console.log (err.name);
  console.log (err.message);
}

There is a third optional parameter, finally. It will happen regardless of whether the try block succeeds or the catch block is called. This gives the option of doing cleanup, closing database connections and doing other cleanup tasks your code needs to complete.

In the example below, the script will always log All Done, regardless if the JSON parsing was successful or not.

try {
  // convert the text representation to JS object
  let user = JSON.parse(json);
  // Log the results to console
  console.log (user.id); // 007
  console.log (user.firstName);  // James
  console.log (user.lastName); // Bond
} catch(err) {
  console.log ('I\'m sorry Dave, I can\'t do that ');
  console.log (err.name);
  console.log (err.message);
} finally {
  //This will always execute regardless
  console.log ('All Done');
}

Brief Introduction to Async Code

Async (short for asynchronous) code will execute without blocking rendering and return a value when the code in the async block finishes.

Contrast the definition of async with the synchronous code we’ve been working so far where all statements are executed in the order they appear.

The example below uses both sync (synchronous) and async (asynchronous) code to illustrate the difference. The console log statements outside of fetch will execute in the order they appear in the document.

console.log ('Starting');
fetch('https://s3-us-west-2.amazonaws.com/s.cdpn.io/32795/coffee2.png') // 1
  .then((response) => {
    if (response.ok) {
      console.log('It worked :)')
      return response.blob(); // 2
    } else {
      throw new Error('Network response was not ok.' );
    }
  })
  .then((myBlob) => {
    let objectURL = URL.createObjectURL(myBlob); // 3
    let myImage = document.createElement('img'); // 4
    myImage.src = objectURL; // 5
    document.body.appendChild(myImage); // 6
  })
  .catch((error) => {
    console.log(
      'There has been a problem with your fetch operation: ',
      error.message
    );
  });
console.log ('All done!');

The fetch function and it’s associated chain uses the Promise API (discussed in more detail in a later document) to get a resource from the network and perform one or more tasks. It won’t make the script wait until it’s done but will continue working in the background until it completes.

The async nature of the Fetch API produces unexpected results. The sequence of messages logged to console:

  1. Starting
  2. All done!
  3. It worked 🙂

The messages from both console.log calls outside fetch appear in their document order but the message signaling success doesn’t appear until the chain started with fetch completes.

Javascript is at its most basic a synchronous, blocking, single-threaded language; Only one operation can be in progress at a time.

Whether we want to run code synchronously or asynchronously will depend on what we’re trying to do.

There are times when we want things to load and happen right away. The callback for a click event handler is a good example.

If we’re running an expensive operation like querying a database and using the results to populate templates then we want to push this off the main thread and complete the task asynchronously.

Performance Timing

There are times when it’s good to have a sense of how long your code takes to execute. Maybe you’re troubleshooting performance or you’re concerned that parts of your code are not as fast as they could be.

In the beginning, you could do something like this:

function myHeavyTask() {
  for (let i = 1; i < 11; i++) {
    console.log(i, i*2, i*3, i*Math.sqrt(2), i*Math.sqrt(3));
  }
};

let startDate = new Date().getTime();
myHeavyTask();
let endDate = new Date().getTime();
let duration = (endDate - startDate);
console.log(duration / 1000 + ' milliseconds');

But it’s cumbersome and error-prone.

The User Timing API allows for custom logging of activity in your page. It also provides a series of predefined events that we can measure against.

The basic tools that we get from the User Timing API provide two tools we can use to measure performance

  • performance.mark gives us a way to indicate when an event has started or finished
  • performance.measure creates a result between two given marks

Different APIs and specifications give you different performance events that you can capture and use as starting points for measurements.

  • domLoading fires the browser is about to start parsing the first received bytes of the HTML document
  • domInteractive triggers when the browser has finished parsing all of the HTML and DOM construction is complete
  • domContentLoaded marks the point when both the DOM is ready and there are no stylesheets that are blocking JavaScript execution – we may be able to construct the render tree.
    • Many JavaScript frameworks wait for this event before they start executing their own logic. For this reason, the browser captures the EventStart and EventEnd timestamps to allow us to track how long this execution took.
  • domComplete indicates all of the processing is complete and all of the resources on the page (images, etc.) have finished downloading
  • loadEvent as the final step in every page load the browser fires an onload event that can trigger additional application logic

The HTML specification dictates specific conditions for each and every event: when it should be fired, which conditions should be met, and so on. We’ll focus on a few key milestones related to the critical rendering path:

  • domInteractive marks when DOM is ready
  • domContentLoaded typically marks when both the DOM and CSSOM are done loading
    • If there is no parser blocking JavaScript then DOMContentLoaded will fire immediately after domInteractive
  • domComplete marks when the page and all of its subresources are ready.

The image below illustrates some of the events available and at what point in the load process they happen.

Different events triggered

Taken from Measuring the Critical Rendering Path

so we can use some of these measurements to create more descriptive names for the events that happen during the page rendering process.

The example below shows how we can measure some events and log the results to the console. It performs the following tasks

  1. Set a variable to hold windows.performance.timing
  2. Measure from when content begins to load to DomInteractive
  3. Measure from when content begins to load to when the domContentLoaded event start
  4. Measure from when content begins to load to when it’s complete
  5. Log the results to the console
let t = window.performance.timing; // 1
let interactive = t.domInteractive - t.domLoading; // 2
let dcl = t.domContentLoadedEventStart - t.domLoading; // 3
let complete = t.domComplete - t.domLoading; // 4
console.log('interactive: ' + interactive / 1000 +  ' seconds'); // 5
console.log('dcl: ' + dcl / 1000 + ' seconds'); // 5
console.log('complete: ' + complete / 1000 + ' seconds'); // 5

Furthermore, we can create custom marks to measure our code’s performance. This is more complex than using the pre-defined benchmarks.

We’ll use the following function as the starting point for explaining performance.

function myFunction(val) {
  for(i = 0; i < val; i++) {};
  console.log('done');
}

We wrap our function executions in two performance.mark with a name. The name is important for the next step. In this case, we’ve named them startTime and endTime.

performance.mark('startTime1');
myFunction(10000);
performance.mark('endTime1');

performance.mark('startTime2');
myFunction(100000);
performance.mark('endTime2');

The next step is to create a measure using performance.measureto display the duration of a performance event.

This method takes three arguments:

  • The name of the measure
  • The beginning mark name
  • The ending mark name

Using the marks we defined in the previous section I’ve created two measures; one for each set of marks.

performance.measure('duration1', 'startTime1', 'endTime1');
performance.measure('duration2', 'startTime2', 'endTime2');

So far we’ve worried about setting up the performance instrumentation for our scripts, now we can look at how to report it using performance.getEntriesByType.

The first example finds all the performance entries of type mark, loops through them and logs their start time.

The second example does the same thing for all measures (performance entries of type measure).

These are trivial examples with few entries. For large codebases, this becomes significantly more useful.

We can also get entries by name, which would allow us to identify individual marks or measures to do further analysis of the application’s performance.

// Grab all the mark entries
perfMarks = performance.getEntriesByType('mark');
// For each mark log name and duration to console
for (i = 0; i < perfMarks.length; i++) {
  console.log(`Name: ${perfMarks[i].name}
  Start Time: ${perfMarks[i].startTime}`)
}

// Grab all the measure entries
perfMeasures = performance.getEntriesByType('measure');
// For each measure log name and duration to console
for (i = 0; i < perfMeasures.length; i++) {
  console.log(`
    Name:  ${perfMeasures[i].name}
    Duration: ${perfMeasures[i].duration}`);
}

Finally, we need to clean up all the marks and measures we create. This is good practice to make sure that we don’t get stale data for our measurements.

performance.clearMarks();
performance.clearMeasures();

Links and Resources