Juliana Louback, software eng.

NLP: Part-Of-Speech Tagging with CYK

Sat, 22 Oct 2016 10:06:52 +0000

During my MSc program, I was lucky to squeeze into Michael Collin’s NLP class. We used his Coursera course as part of the program, which I’d highly recommend.

Recently I decided to review my NLP studies and I believe the best way to learn or relearn a subject is to teach it. This is the second in a series of 4 posts with a walk-through of the algorithms we implemented during the course. I’ll provide links to my code hosted on Github.

Disclaimer: Before taking this NLP course, the only thing I knew about Python was that ‘it’s the one without curly brackets’. I learned Python on the go while implementing these algorithms. So if I did anything against Python code conventions or flat-out heinous, I apologize and thank you in advance for your understanding. Feel free to write and let me know.

The Concept

The Cocke–Younger–Kasami algorithm is used to parse context free grammars. A context free grammar (CFG) contains a set of rules that describe a [formal] language. For example, for the English language we would set a rule that says an article is always followed by a noun or an adjective(s) + noun combo. A context free grammar has to have enough rules to cover all the possible variations in the language. The rules can be one -> one, one -> many or one -> stop. There are two types of building blocks for these rules; non-terminals and terminals. A non-terminal contains one or more terminals, for example, a nominal phrase. A nominal phrase contains a noun and maybe also an article and/or adjectives. The noun, article, and adjectives are all terminals. So for the sentence “The dog barked.” we could establish the following rules:

S -> NP VP

NP -> DET N

VP -> V

DET -> The

NOUN -> dog

VERB -> barked

Where S is the sentence (or rather, the start symbol of the sentence), NP is the nominal phrase, VP is the verbal phrase, DET a determinant, NOUN a noun, VERB a verb.

These rules can create a part-of-speach parse tree for any given sentence providing an understanding of how different parts of the sentence interact with and affect each other. This can be useful for machine translation, information extraction and other scenarios. However, there are cases where there is more than one grammatically correct parsing option. Take the sentence “He fed her dog food”. This could be interpreted as (A) A man feeding a woman’s dog some food or (B) A man feeding a woman some dog food. A human would expect option (A) to be the correct interpretation (hopefully?), but would an algorithm?

The CYK algorithm enriches the CFG by adding a parameter q(X -> Y) that could be described as the probability of choosing rule X -> Y (as opposed to some other rule X -> Z) given that we are currently at X. The probability of a parse tree is the product of all q parameters for the rules that compose the tree. Given these q parameters, the CYK algorithm chooses the most likely parse tree out of all possible parse trees. Note that a parse tree has a cascading effect, you may chose a rule X -> Y because it has a higher q parameter than X -> Z but then the subtree starting from Y may not have a valid rule to fit the sentece (q parameter 0) or is much less likely than a subree starting from Z. In that case, the rule X -> Z would have been the best choice. In other words, it’s not enough to just use a “greedy” technique and always choose the rule with the highest q parameter.

The Requirements

Training data: The file parse_train.dat provided by prof. Michael Collins (from the Wall St. Journal corupus) has a series of sentences already tagged with their respective POS and parsed into a tree. Each line is one tree, each level of a tree separated by square braquets.

Development data: The file parse_dev.dat provided by prof. Michael Collins has a list of sentences (one per line).

Script for rule count: The file count_cfg_freqs.py provided by prof. Michael Collins contains a script to produce the rule counts.

Training data rule count: Run python count cfg freqs.py parse train.dat > cfg.counts to obtain the list of rule counts from the training data. The format is count ruleType rule separated by tabs. Rules can be unary (example: V -> run), binary (example: Sentence -> NP VP) or nonterminal (list of acronyms representing nonterminals, example: NP).

The Algorithm

Python code: cyk.py Usage: python cyk.py cfg_vert.counts parse_dev.dat cfg.counts > [output_file] Summary:

Optional Preprocessing

Re-label words in training data with frequency < 5 as ‘RARE’ - This isn’t required, but useful to increase precision. If used, perform Step 1 (see below) then re-label the words to redo Step 1.

Python code: relabel_rare.py

Usage: python relabel_rare.py [input_file]

Pseudocode:

Uses Python Counter to obtain word counts in [input_file]; removes all word-count pairs with count < 5, store remaining pairs in a dictionary named rare_words.
Iterates through each line in [input file], checks if word is in rare words dictionary, if so, replaces word with RARE.

Step 1. Get binary, unary and terminal rule counts

Python code: count_cfg_freq.py

Pseudocode: Iterate through each line in the parse_train.dat file and obtain counts for binary rules (example: S -> NP VP), unary rules (DET -> the) and nonterminals (examples: ADJ, VERB, etc). The output has one count+rule per line, each item of the line is separated by spaces. The line format is

{count} NONTERMINAL {X} in the case of listing a nonterminal.

{count} UNARY {X} {Y} in the case of a unary rule X -> Y

{count} BINARY {X} {Y} {Z} in the case of a binary rule X -> Y Z

The abbreviations used are pretty intuitive. It’s not entirely necessary to understand the meaning of said abbreviations, but here’s a list of the most common nonterminals:

S start of sentence

NP nominal phrase

VP verbal phrase

NOUN noun

VERB verb

ADJ adjective

ADV adverb

PP preposition

DET determinant

The script listed above (method get_counts) will return 3 dictionaries. The dictionary with terminal counts has a string as key and an int as value. The key is the terminal symbol, value is the count. For example, {“DET”:5623}. The unary counts have a string as key and a dictionary with {string:int} as value. The primary key is the unary symbol; the value has the actual word as key and it’s count. For example, {“DET”:{“the”:4833}}. Finally the binary counts are also a string as key and a dictionary with {string:int} as value. The primary key is still the X non-terminal in the binary rule X -> Y Z; the value has a dictionary with “Y Z” (the right had part of the binary rule) with the respective count as value. For example, {“S”:{“NP VP”:101788}}.

It’s not as bad as it sounds.

Step 2. Context-Free Grammar to Probabilistic Context-Free Grammar: calculate rule parameters

Python code: cyk.py

Get the 3 dictionaries with terminal, unary and binary rule counts. The following steps are done for each line (sentence) in the development data. Basically this is where it all happens, after all you want to parse a sentence with POS tags. The amount of sentences is irrelevant; just useful to measure accuracy and performance.
Create two matrices that are nx(n+1) where n is the length of the sentence (number of words and punctuation symbols). Both matrices contain dictionaries. One will be used to store probabilities of a unary rule (example: {“DET”:0.28}) the other will contain the ‘backpointer’, rather the actual word the unary rule is being applied to (example: {“DET”:”the”}). These probabilities and backpointers will be stored on the diagonal starting at [0][1] of their respective matrices.
Here’s the part that gets slightly confusing to read the code. Starting from the upper left corner (i.e. coordinates, 0 and 1 if your language starts with 0 as the first position of an array), go adding binary rules to the diagonal of probabilities/backpointers filling up the upper right triangle of the matrix:
- For all the binary rules in your list (dictionary of counts) check if the non-terminal beside the current position in the probability matrix (say at [0][1]) and the one “bellow” on the diagonal (in this case, [1][2]) are existing keys in the probability matrix. Note that if you are further above the diagonal starting at [0][1] built in step 2, you will have more options of combinations. For example, when filling in [0][3], you can check [0][1] and [1][2], [0][2] and [1][2], [0][2] and [1][3], [1][2] and [0][3].
- If so, calculate the binary rule probability as (binary X -> Y Z rule count / non-terminal count of X) * (probability of unary rule for Y) * (probability of unary rule for Z)
- If there hasn’t yet been a key-value pair stored in the probability matrix [0][2] with the key X, set the value of key X to the probability above. If not, check if the current binary rule has higher probability and if so, update the X entry. Update the backpointer matrix at [0][2] to contain {“X”:”Y Z”} (if it’s a new key or if the current probability is the highest yet).
The upper right corner of the matrix should have an entry with the sentence starter symbol S. This dictionary would only have one entry because it would use the highest probability rule (it adds up). If it doesn’t (say your set doesn’t use starter symbols) it’s no issue, just ‘trace’ each value in the dictionary (recursively) to see which alternative has the highest probability.
Return the parse tree with highest probability.

Step 3 is easier with examples. Take the sentence “The dog barked”. So in step 2 you’ll probably have entered the pair {“DET”:”the”} in the backpointer matrix at [0][1] and {“DET”:0.12} (random probability value used here) in the probability matrix at [0][1]. You’d also enter {“NOUN”:”dog”} and {“NOUN”:0.2} at positions [1][2] in each matrix. So take the rule NP -> ADJECTIVE NOUN. Although DET and NOUN will likely have been matched to “the” and dog” and would have been included in the probability/backpointer matrix, the non-terminal ADJECTIVE wouldn’t match to “the” so this binary rule is a no-go. Now for the rule NP -> DET NOUN we have a match, so we add an entry {“NP”:0.23} to the probability matrix at [0][2] and {“NP”:”DET NOUN”} to position [0][2] in the backpointer matrix. We fill up the upper right triangle of the matrix.

The great thing is, this algorithm can tolerate tagging errors. Let’s say once or a few times the word “the” was tagged as “ADJECTIVE”. So in the backpointer matrix [0][1] you would store the original entry {“DET”:”the”} but also {“ADJECTIVE”:”the”}. In the probability matrix at [0][1] you’d store {“DET”:q₁} and {“ADJECTIVE”:q₂} where q₁ and q₂ are the probabilities of the unary rules DET -> the and ADJECTIVE -> the. Now it’s very likely that q₁ > q₂ (unless you have a really badly tagged training set). When checking which binary rule that starts with NP is best, you’ll probably run into NP -> DET NOUN and NP -> ADJECTIVE NOUN. These two binary rules could both be equally likely in their own right, let’s say they both equal b.

p(NP -> DET NOUN) == p(NP -> ADJECTIVE NOUN) == b

p(NOUN -> dog) == c

p(DET -> the) == q1

p(ADJECTIVE -> the) == q2

and q1 > q2

then b * q1 * c > b * q2 * c

So you will chose the correct option NP -> DET NOUN as the subtree root at [0][2].

Evaluation

Prof. Michael Collins provided a script to pretty print the parse trees and a performance evaluation script.

Usage: python pretty_print_parse.py [prediction_file] python eval_parser.py parser_dev.key [prediction_file]

NLP: Viterbi Named-Entity Tagger

Sun, 10 Jan 2016 10:06:52 +0000

During my MSc program, I was lucky to squeeze into Michael Collin’s NLP class. We used his Coursera course as part of the program, which I’d highly recommend.

Recently I decided to review my NLP studies and I believe the best way to learn or relearn a subject is to teach it. This is one in a series of 4 posts with a walk-through of the algorithms we implemented during the course. I’ll provide links to my code hosted on Github.

The Concept

To quote Wikipedia, “Named-entity recognition (I’ve always known it as tagging) is a subtask of information extraction that seeks to locate and classify elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quanitites, monetary values, percentages, etc.”

For example, the algorithm receives as input some text

“Bill Gates founded Microsoft in 1975.”

and outputs

“Bill Gates[person] founded Microsoft[organization] in 1975[date].”

Off the top of my head, some useful applications are document matching (ex. a document containing Gates[person] may not be on the same topic as one containing gates[object]) and query searches. I’m sure there are lots more, if you check out Collin’s Coursera course he may discuss this in greater depth.

The Requirements

Development data: The file ner_dev.dat provided by prof. Michael Collins has a series of sentences separated by an empty line, one word per line.

Training data: The file ner_train.dat provided by prof. Michael Collins has a series of sentences separated by an empty line, one word and tag per line, speparated by a space.

Word-tag count data: The file ner.counts has the format [count] [type of tag] [label] [word]. The tags used are RARE, O, I-MISC, I-PER, I-ORG, I-LOC, B-MISC, B-PER, B-ORG, B-LOC. The tag O means it’s not an NE. This file is generated by count_freqs.py, a script provided by prof. Michael Collins. Run count_freqs.py on the training data ner_train.dat

The Algorithm

Python code: viterbi.py Usage: python viterbi.py ner.counts ngram.counts [input_file] > [output_file] Summary: The Viterbi algorithm finds the maximum probability path for a series of observations, based on emission and transition probabilities. In a Markov Process, emission is the probability of an output given a state and transition is the probability of transitioning to the state given the previous states. In our case, the emission parameter e(x|y) = the probability of the word being x given you attributed tag y. If your training data had 100 counts of ‘person’ tags, one of which is the word ‘London’ (I know a guy who named his kid London), e(‘London’|’person’) = 0.01. Now with 50 counts of ‘location’ tags, 5 of which are ‘London’, e(‘London’|’location’) = 0.1 which clearly trumps 0.01. The transition parameter q(y_i | y_i-1, y_i-2) = the probability of putting tag y in position i given it’s two previous tags. This is calculated by Count(trigram)/Count(bigram). For each word in the development data, he Viterbi algorithm will associate a score for a word-tag combo based on the emission and transition parameters it obtained from the training data. It does this for every possible tag and sees which is more likely. Clearly this won’t be 100% correct as natural language is unpredictable, but you should get pretty high accuracy.

Optional Preprocessing

Re-label words in training data with frequency < 5 as ‘RARE’ - This isn’t required, but useful. Re-run count_freqs.py if used.

Python code: label_rare.py

Usage: python label_rare.py [input_file]

Pseudocode:

Uses Python Counter to obtain word counts in [input_file]; removes all word-count pairs with count < 5, store remaining pairs in a dictionary named rare_words.
Iterates through each line in [input file], checks if word is in rare words dictionary, if so, replaces word with RARE.

Step 1. Get Count(y) and Count(x~y)

Python code: emission_counts.py

Pseudocode:

Iterate through each line in ner.counts file and store each word-label-count combo in a dictionary count_xy and update the dictionary of count_y. For example count_xy[Peter][I-PER] returns the number of times the word ‘Peter’ was labeled ‘I-PER’ in the training data and count_y[I-PER] the total number of ‘I-PER’ tags. The dictionary count_y contains 8 items, one for each label ( RARE , O, I-MISC, I-PER, I-ORG, I-LOC, B-MISC, B-PER, B-ORG, B-LOC);
Return count_xy, count_y

Step 2. Get bigram and trigram counts

Python code: transition_counts.py

Pseudocode:

Iterate through each line in the n-gram_counts file
If the line contains ’2-GRAM’ add an item to the bigram_counts dictionary using the bigram (two space-separated labels following the tag type ‘2-gram’) as key, count as value. This dictionary will contain Count(y_i-2,y_i-1).
If the line contains ’3-GRAM’, add an item to the trigram_counts dictionary using the trigram as key, count as value. This dictionary will contain Count(y_i-2, y_i-1, y_i).
Return dictionaries of bigram and trigram counts.

Step 3. Viterbi

(For each line in the [input_file]):

If the word was seen in training data (present in the count_xy dictionary), for each of the possible labels for the word:
Calculate emission = count_xy[word][label] / float(count_y[label]
Calculate transition = trigram_counts[trigram])/float(bigram_counts[bigram] Note: y_i-2 = *, y_i-1 = * for the first round
Set probability = emission x transition
Update max(probability) and arg max if needed. 2 If the word was not seen in the training data:
Calculate emission = count xy[RARE][label] / float(count y[label].
Calculate q(y_i y_i-1, y_i-2) = trigram counts[trigram])/float(bigram counts[bigram]. Note: y_i-2 = ∗, y_i-1 = ∗ for the first round
Set probability = emission × transition
Update max(probability) if needed, arg max = RARE
Write arg max and log(max(probability)) to output file.
Update y_i-2, y_i-1.
Update y_i-2, y_i-1.

Evaluation

Prof. Michael Collins provided an evaluation script eval_ne_tagger.py to verify the output of your Viterbi implementation. Usage: python eval_ne_tagger.py ner_dev.key [output_file]

PaperTrail - Powered by IBM Watson

Sat, 14 Nov 2015 10:06:52 +0000

On the final semester of my MSc program at Columbia SEAS, I was lucky enough to be able to attend a seminar course taught by Alfio Gliozzo entitled Q&A with IBM Watson. A significant part of the course is dedicated to learning how to leverage the services and resources available on the Watson Developer Cloud. This post describes the course project my team developed, the PaperTrail application.

Project Proposal

Create an application to assist in the development of future academic papers. Based on a paper’s initial proposal, Paper Trail predicts publications to be used as references or acknowledgement of prior art and provides a trend analysis of major topics and methods.

The objective is to speed the discovery of relevant papers early in the research process, and allow for early assessment of the depth of prior research concerning the initial proposal.

Meet the Team

Wesley Bruning, Software Engineer, MSc. in Computer Science

Xavier Gonzalez, Industrial Engineer, MSc. in Data Science

Juliana Louback, Software Engineer, MSc. in Computer Science

Aaron Zakem, Patent Attorney, MSc. in Computer Science

Prior Art

A significant amount of attention has been given to this topic over the past few decades. The table below shows the work the team deemed most relevant due to recency, accuracy and similarity of functionality.

The variation in accuracy displayed is a result of experimentation with different dataset sizes and algorithm variations. More information and details can be found in the prior art report.

The main differential of PaperTrail is providing a form of access to the citation prediciton and trend analysis algorithm. With the exception of the project by McNee et al., these algorithmns aren’t currently available for general use. The application on researchindex.net is open to use but its objective is to rank publications and authors for given topics.

Algorithm

Citation Prediction: PaperTrail builds on the work done by Wolski’s team in Fall 2014. This algorithmn builds a reference graph used to define research communities, with an associated vector of topic scores generated by an LDA model. The papers in each research community are then ranked by importance within the community with a custom ranking algorithm. When a target document is given to algorithm as input, the LDA model is used to generate a vector of topics that are present in the document. The communities with the most similar topic vectors are selected and the publications within these communities with highest rank and greatest similarity to the input document are recommended as references. A more detailed description can be found here.

Trend Analysis: Initially, the idea was to use the AlchemyData News API to obtain statistics pertaining to the amount of publications on a given topic over time. However, with the exception of buzz-words (i.e. ‘big data’), many more specialized topics appeared very infrequently in news articles, if at all. This isn’t entirely surprising given the target audience of PaperTrail. As a work around, we use the Alchemy Language API to extract keywords from the abstracts in the dataset, in addition to relevance scores. The PaperTrail database could then be queried for entry counts for a given year and keyword to provide an indication of publication trends in academia. Note that the Alchemy Language API extracts multiple-word ‘keywords’ as well as single words.

Data

To maintain consistency with Wolski’s project, we are using the DBLP data as made available on aminer.org. The DBLP-Citation-network V5 dataset contains 1,572,277 entries; we are limited to the use of entries that contain both abstracts and citations, bringing the dataset size down to 265,865 entries.

Architecture

A high-level visualization of the project architecture is displayed below. Before launching PaperTrail, it’s necessary to train Wolski’s algorithm offline. Currently any documentation with regard to the performance of said algorithm is unavailable; the PaperTrail project will include an evaluation phase and report the findings made.

The PaperTrail app and database will be hosted on the Bluemix Platform.

Status Report

Phases completed:

Project design
Prior art research
Data cleansing
Development and deployment of an alpha version of the PaperTrail app

Phases under development:

Algorithm training and evaluation
Keyword extraction
MapReduce of publication frequency by year and topic
Data visualization component

Getting into Google - The Timeline

Mon, 05 Jan 2015 10:06:52 +0000

As part of my Brazil Scientific Mobility Masters Cohort Program (what a mouthful), I have to engage in research or an internship related to my field of study during the summer - no 3 month vacation. This isn’t as terrible as it sounds, particularly if you consider my summer will be spent working at Google.

You don’t get that much desired offer letter before going through quite a battery of “entrance exams”. The selection process isn’t easy - or at least I didn’t think so. I thought I’d write about somthing I concerned myself much about: the admission timeline. Hopefully this can be of help to people currently going through the process or for those who’d like to know what they’re getting into.

The Timeline

You’ll hear about this in most blogs. The intern selection process is usually divided into three parts: phone interview, technical interview(s), matching. From what I researched, my process took longer than usual. This may be because it happened over the holidays/end of year. This was nerve-wracking for me, so I thought I’d share my case so others worry less. I used my email history to get dates, so this should be pretty accurate.

0. The First Application: As I was looking up old emails, I found an email dated ~ Feb 2013, the end of my junior year in undergrad, kindly informing me that I was not chosen for an internship at Google. I’d forgotten I’d applied way back when. But they said they’d keep my resume on file and turns out they actually did. One year later in June 2014, I get an email from a recruiter. She wanted to interview me for a full time position. My resume included my June 2014 expected graduation so I’m guessing that’s why I got a callback.

By that time I had already accepted a grant funding my MS at Columbia, so I explained I would only be free for a summer 2015 internship. My recruiter was great about it, said she’d recommend me to the U.S. internship recruiting department, but suggested we go ahead and have a phone interview. Lesson here, getting a ‘no’ doesn’t mean you’re no good. It could be the wrong time, or maybe you just need to work a bit more, learn a bit more and try again.

1. Phone Interview - June 2014: The phone interview is no biggie. A few tech-related questions, but all the kind you can answer in one sentence. Mostly to get to know you, your experience, interests, etc. ~30 min.

After the call, I wrote thanking my recruiter for the opportunity. She wrote back that day giving me prep material for the technical interview, which I assume meant I passed that part. As I was only applying for summer 2015, she informed me she’d be in touch after the summer which is when the technical interviews would take place.

2. The Second Application - September 2014: In mid-September, the Google recruiter contacted me again, asking for an updated transcript, resume, the works. I think I applied around then to a summer 2015 software engineering internship. I don’t remember if I was instructed to, or if I simply did it on a hunch.

By the end of the month (26th to be exact), I got an email from another recruiter scheduling the technical interview. The interviews were to take place about a month later. The recruiter gave me a window of time to have the interviews, I gave them several days I’d be free and he chose one.

3. The Technical Interviews - October 2014: Both my technical interviews took place on the afternoon October 22nd. I got a very sweet, “you can do it” email from my recruiter with tips and helpful suggestions. I should take the time to mention how incredibly nice all the recruiters I interacted with were. They really only get the best people to work at Google. It was an A+ experience.

The interviews were held over a video call, each 1 hour long, all coding done on Google Docs. This done, I patiently waited, or at least I tried very hard to be patient. I wasn’t. Not my forté. I refreshed my inbox every 5-10 minutes.

4. The Technical Interview Turnout - November 2014: Near the date of the technical interviews, I was informed I would get an answer in 1-2 weeks. I interviewed on a Wednesday, October 22nd; I got the call back on a Monday, November 3rd. Almost exactly 2 weeks. Felt like a year. If they’d waited another day I think I would’ve developed an ulcer I was so anxious.

It was great to hear I passed the technical interviews, but as they say, it’s not over until the fat lady sings. The metaphorical ‘fat lady’ was not singing. Once you pass the technical interviews, you go into a matching phase. You fill out a candidate profile with your skills, location preferences, strengths and interests, then the teams needing an intern contact your recruiter should you seem like a good fit.

5. The Matching Phase - November 2014 to January 2015: I was told that if you don’t get matched in circa 6 weeks, you bow out of the intern selection process. This isn’t an urban legend, one of my fellow students did make it to the matching phase but didn’t get matched (he did land a killer offer later on though). He’s an excellent engineer, but there is such a thing as wrong place and/or time. Rumor has it he received a full-time job offer from Google upon graduation.

I learned I passed the technical interview part in the first week of November and on the second week of December I received news of a possible team match. That was more or less 6 weeks, yikes. We scheduled a call between me and a member of the team. This isn’t a technical interview, just a ‘getting to know you’ talk, to see if you’re a good fit. I think a day or two later I heard back from the team, they wanted me! The team seemed great, I got to meet them when I flew out to the West Coast, but the project itself was not in a domain I was particularly experienced in. I asked my recruiter if I could wait a bit before making a decision. Bold move, Cotton.

Apparently the 6 week deadline isn’t necessarily a hard and fast rule. About three weeks later in the beginning of January I heard about another possible match. This time the project was perfectly aligned to both my strengths and my interests. I could not hide how interested I was in working on the project. The kind of thing I’d do for free, heck, I’d pay them to let me work on the project. A couple days later, I heard back from the team lead confirming their interest in working with me.

6. The Offer Letter - January 2015: On January 17th, 2015, I got that awesome offer letter, then spent the whole day smiling like an idiot. Actually that went on much longer than that, now and then I remember I’m heading there soon (in two weeks) and I smile stupidly again.

I had two weeks to accept/rejct the offer, needless to say I signed it the same day. So that was that. I think usually the phone interview is usually right before the technical interviews. Even so, my entire hiring process took about 3 months. If this happens to you, don’t take it as a bad sign. I hear good things come to those who wait.

JSCommunicator at xTupleCon 2014

Thu, 30 Oct 2014 10:06:52 +0000

Two weeks ago I left NYC for a day trip to Norfolk, Virginia, to attend xTupleCon 2014. For those who don’t know, xTuple is an incredibly reknown open source Enterprise Resource Planning (ERP) software. If you’ve been following this blog, you might recall that during my participation in the Google Summer of Code 2014, I wrote a beta JSCommunicator extension for xTuple (to see how I went about doing that, look up Kickstarting the JSCommunicator-xTuple extension).

Now, I wasn’t flying down to Virginia on the eve of my first grad school midterms (gasp!) for fun - although I will admit, I enjoyed myself a fair amount. My GSoC mentor, Daniel Pocock was giving a talk about WebRTC and JSCommunicator at xTupleCon and invited me to participate. So that was the first perk of going to xTupleCon, I got to finally meet my GSoC mentor in person!

During the presentation, Daniel provided a high level expanation of WebRTC and how it works. WebRTC (Web Real Time Communications) enables real time transmission of audio, video and data streams through browser-to-browser applications. This is one of the many perks of WebRTC; it doesn’t require installation of plugins, making its use less vulnerable to security breaches. Websockets are used for the signalling channel and the SIP or XMPP can be used as the signaling protocol - in JSCommunicator, we used SIP. What was also done in JSCommunicator and can be done in other applications is to use a library to implement the signaling and SIP stack. JSCommunicator uses (and I highly reccomend) JsSIP.

The basic SIP architecture is comprised of peers and a SIP Proxy Server that supports SIP over Websockets transport:

If you have users behind NAT, you’ll also need a TURN server for relay:

In both cases, setup is not too difficult, particularly if using reSIProcate which offers both the SIP proxy and the TURN server. Daniel Pocock has an excellent post on how to setup and configure your SIP proxy and TURN server.

With regard to JSCommunicator, it is a generic telephone in HTML5 which can easily be embedded in any web site or web app. Almost every aspect of JSCommunicator is easily customizable. More about JSCommunicator setup and architecture is detailed in a previous post.

The JSCommunicator-xTuple extension can be installed in xTuple as an npm package (xtuple-jscommunicator). It is still at a very beta - or even pre-beta - stage and there are various limitations; the configuration must be hard-coded and dialing is done manually as opposed to clicking on a contact. Some of these ‘limitations’ are features are on the wish list for future work. For example, some ideas for the next version of the extension are a click to dial from CRM records functionality and to bring up CRM records for an incomming call. Additionaly, the SIP proxy could be automatically installed with the xTuple server installation if desired.

We closed the presentation with a live demo during which I made a video call from a JSCommunicator instance embedded in freephonebox.net to the JSCommunicator xTuple extension running on Daniel’s laptop. Despite the occasionally iffy hotel Wifi, the demo was a hit - at one point I even left the conference room and took a quick walk around the hotel and invited other xTupleCon attendees to say hello to those in the room. The audience’s reception was more enthusiastic than I anticipated, giving way to a pretty extensive Q&A session. It’s great to see more and more people interested in WebRTC, I can’t emphasize enough what a useful and versatile tool it is.

Here’s an ‘action shot’ of part of the xTuple WebRTC presentation:

Debconf 2014 and How I Became a Debian Contributor

Mon, 01 Sep 2014 10:06:52 +0000

Part 1 - Debconf 2014

This year I went to my first Debconf, which took place in Portland, OR during the last week of August 2014. All in all I have to rate my experience as very enlightening and in the end quite fun.

First of all, it was a little daunting to go to a conference in 1 - A city I’d never been to before; 2 - A conference with 300+ people, only 3 of which I knew and even then I only knew them virtually. Not to mention I was in the presence of some extremely brilliant and known contirbutors in the Debian community which was somewhat intimidating. Just to give you an idea, Linus Torvalds showed up for a Q&A session last Friday morning! Jealous? Actually I missed that too. It was kind of a last minute thing, booked for coincidentally the exact time I’d be flying out of Portland. I found out about it much too late. But luckily for me and maybe you, the session was filmed and can be seen here. Isn’t that a treat?

Point made, there are lots of really talented people there, both techies and non-techies. It’s easy to feel you’re out of league, at least I did. But I’d highly encourage you to ignore such feelings if you’re ever in the same situation. Debian has been built on for a long time now, but although a lot has been done, a lot still needs to be done. The Debian community is very welcoming of new contributors and users, regardless of the level of expertise. So far I haven’t been snubbed by anyone. To the contrary, all my interactions with the Debian community members has been extremely positive.

So go ahead and attend the meetings and presentations, even if you think it’s not your area of expertise. Debconf was organized (or at least this one was) as a series of talks, meet ups and ad hoc sessions, some of which occured simultaneously. The sessions were all about different components of the Debian universe, from presenting new features to overviews of accomplishments to discussing issues and how to fix them. A schedule with the location and description of each session was posted on the Debconf wiki. Sometimes none of the sessions at a certain time was on a topic I knew very much about. But I’d sit in anyways. There’s no rule to attending the sessions, no ‘minimum qualifications’ required. You’ll likely learn something new and you just might find out there is something you can do to contribute. There are also hackathons that are quite the thing or so I heard. Or you could walk about and meet new people, do some networking.

I have to say networking was the highlight of the Debconf for me. Remember I said I knew about 3 people who were at the conference? Well, I had actually just corresponded with those people. I didn’t really know them. So on my first day I spent quite some time shyly peeking at people’s name tags, trying to recognize someone I had ‘met’ over email or IRC. But with 300 or so people at the conference, I was unsuccessful. So I finally gave up on that strategy and walked up to a random person, stuck out my hand and said, “Hi. My name is Juliana. This is my first Debconf. What’s your name and what do you do for Debian?” This may not be according to protocol, but it worked for me. I got to meet lots of people that way, met some Debian contributos from my home country (Brazil), some from my current city (NYC), and yet others that had similar interests as I do who I might work with in the near future. For example, I love Machine Learning, I’m currently beginning my graduate studies on that track. Several Debian contributors offered to introduce me to a well known Machine Learning researcher and Debian contributor who is in NYC. Others had tried out JSCommunicator and had lots of suggestions for new features and fixes, or wanted to know more about the project and WebRTC in general. Also, not everyone there is a super experienced Debian contributor or user. There are a lot of newbies like me.

I got to do a quick 20-min presentation and demo of the work I had done on JSCommunicator during GSoC 2014. Oh my goodness that was nerve-wracking, but not half as painful as I expected. My mentor (Daniel Pocock) wisely suggested that when confronted with a question I didn’t know how to answer, to redirect the question to to the audience. Chances are, there is someone there that knows the answer. If not, it will at least spark a good discussion.

When meeting new people at Debian, a question almost everyone asked is “How did you start working with/for Debian?”. So I thought it would be a good topic to post about.

Part 2 - How I Became a Debian Contributor

Sometime in late October of 2013 (I think) I received an email from one of my professors at UNIRIO forwarding a description of the Outreach Program for Women. OPW is a program organized by the GNOME which endeavors to get more women involved in FOSS. OPW is similar to Google Summer of Code; you work remotely from home, guided by an assigned mentor. Debian was one of the 8 participating organizations that year. There was a list of project proposals which I perused, a few of them caught my eye and these projects were all Debian. I’d already been a fan of FOSS before. I had used the Ubuntu and Debian OS, I’d migrated to GIMP from Photoshop and to Open Office from Microsoft Office, for example. I’d strongly advocated the use of some of my prefered open source apps and programs to my friends and family. But I hadn’t ever contributed to a FOSS project.

There’s no time like the present, so I reached out the the mentor responsible for one of the projects I was interested in, Daniel Pocock. Daniel guided me through making a small contribution to a FOSS project, which serves as a token demonstration of my abilities and is part of the application process. I added a small feature to JMXetric and suggested a fix for an issue in the xTuple project. Actually, I had forgotten about this. Recently I made another contribution to xTuple, it’s funny to see things come full circle. I also had to write a profile-ish description of my experience and how I intended on contributing during OPW on the Debian wiki, if you’d like you can check it out here.

I wouldn’t follow my example to a T, because in the end I didn’t make the OPW selection. Actually, I take that back. The fact I wasn’t chosen for OPW that year doesn’t mean I was incompetent or incapable of making a valuable contribution. OPW and GSoC do not have unlimited resources; they can’t include everyone they’d like to. They receive thousands of proposals from very talented engineers and not everyone can participate at a given moment. But even though I wasn’t selected, like I said, I could still pitch in. It’s good to keep in mind that people usually aren’t paid to contribute to FOSS. It’s usually volunteer based, which I think is one of the beauties of the FOSS community and in my opinion one of the causes of it’s success and great quality. People contribute because they want to, not because they have to.

I will say I was a little disappointed at not being chosen. But after being reassured that this ‘rejection’ wasn’t due to any lack on my part, I decided to continue contributing to the Debian project I’d applied to. I was begining the final semester of my undergraduate studies which included writing a thesis. To be able to focus on my thesis and graduate on time, I’d stopped working temporarily and was studying full time. But I didn’t want to lose practice and contributing to a FOSS project is a great way to stay in coding shape while doing something useful. So continue contributing I did.

It paid off. I gained experience, added value to a FOSS project and I think my previous contributions added weight to the application I later made for GSoC 2014. I passed this time. To be honest, I really wasn’t counting on it. Actually, I was certain I wouldn’t pass for some reason - insecure much? But with GSoC I wasn’t too anxious about it as I was with the OPW application because by then, I was already ‘hooked’. I’d learned about all the benefits of becoming a FOSS contributor and I wasn’t stopping anytime soon. I had every intention of still working on my FOSS project with or without GSoC. GSoC 2014 ended a week ago (August 18th 2014). There’s a list of things I still want to do with JSCommunicator and you can be sure I’ll keep working on them.

P.S. This is not to say that programs like OPW and GSoC aren’t amazing programs. Try it out if you can, it’s really a great experience.

JSCommunicator 2.0 (Beta) is Live!

Thu, 14 Aug 2014 23:06:52 +0000

This is the last week of Google Summer of Code 2014 - all good things must come to an end. To wrap things up, I’ve merged all my work on JSCommunicator into a new version with all the added features. You can now demo the new and improved (or at least so I hope) JSCommunicator on rtc.debian.org!

JSCommunicator 2.0 has an assortment of new add-ons, the most important new features are the Instant Messaging component and the internationalization support.

The UI has been reorganized but we are currently not using a skin for color scheme - will be posting about that in a bit. The idea is to have a more neutral look that can be easily customized and integrated with other web apps.

A chat session is automatically opened when you begin a call with someone - unless you already started a chat session with said someone. Sound alerts for new incoming messages are optional in the config file, visual alerts occur when an inactive chat tab receives a new message. Future work includes multiple user chat sessions and adapting the layout to a large amount of chat tabs. Currently it only handles 6. (Should I allow more? Who chats with more than 6 people at once? 14 year old me would, but now I just can’t handle that. Anyway, I welcome advice on how to go about this. Should we do infinite tabs or if not, what’s the cut-off?)

About internationalization, I’m uber proud to say we currently run in 6 languages! The 6 are English (default), Spanish, French, Portuguese, Hebrew and German. But one thing I must mention is that since I added new stuff to JSCommunicator, some of the new stuff doesn’t have a translation. I took care of the Portuguese translation and Yehuda Korotkin quickly turned in the Hebrew translation, but we are still missing an update for Spanish, French and German. If you can contribute, please do. There are about 10 new labels to translate, you can fix the issue here. Or if you’re short on time, shoot me an email with the translation for what’s on the right side of the ‘=’:

welcome = Welcome,

call = Call

chat = Chat

enter_contact = Enter contact

type_to_chat = type to chat…

start_chat = start chat

me = me

logout = Logout

no_contact = Please enter a contact.

remember_me = Remember me

I’ll merge it myself but I’ll be sure to add you to the authors list.

JSCommunicator - Setup and Architecture

Mon, 11 Aug 2014 15:06:52 +0000

Preface

During Google Summer of Code 2014, I got to work on the Debian WebRTC portal under the mentorship of Daniel Pocock. Now I had every intention of meticulously documenting my progress at each step of development in this blog, but I was a bit late in getting the blog up and running. I’ll now be publishing a series of posts to recap all I’ve done during GSoC. Better late than never.

Intro

JSCommunicator is a SIP communication tool developed in HTML and JavaScript. The code was designed to make integrating JSCommunicator with a website or web app as simple as possible. It’s quite easy, really. However, I do think a more detailed explanation on how to set things up and a guide to the JSCommunicator architecture could be of use, particularly for those wanting to modify the code in any way.

Setup

To begin, please fork the JSCommunicator repo.

If you are new to git, feel free to follow the steps in section “Setup” and “Clone” in this post.

If you read the README file (which you always should), you’ll see that JSCommunicator needs a SIP proxy that supports SIP over Websockets transport. Some options are:

I didn’t find a tutorial for Kamailio setup, I did find one for repro setup. And bonus, here you have a great tutorial on how to setup and configure your SIP proxy AND your TURN server.

In your project’s root directory, you’ll see a file named config-sample.js. Make a copy of that file named config.js. The config-sample.js file has comments that are very instructive. In sum, the only thing you have to modify is the turn_servers property and the websocket property. In my case, debrtc.org was the domain registered for testing my project, so my config file has:

turn_servers: [
	{ server:"turn:debrtc.org" }     
],

Note that unlike the sample, I didn’t set a username and password so SIP credentials will be used.

Now fill in the websocket property – here we use sip5060.net.

websocket: {
    servers: 'wss://ws.sip5060.net',
    connection_recovery_min_interval: 2,
    connection_recovery_max_interval: 30,
  },

I’m pretty sure you can set the connection_recovery properties to whatever you like. Everything else is optional. If you set the user property, specifically display_name and uri, that will be used to fill in the Login form and takes preference over any ‘Remember me’ data. If you also set sip_auth_password, JSCommunicator will automatically log in.

All the other properties are for other optional functionalities and are well explained.

You’ll need some third-party javascript that is not included in the JSCommunicator git repo. Namely jQuery version 1.4 or higher and ArbiterJS version 1.0. Download jQuery here and ArbiterJS here and place the .js files in your project’s root directory. Do make sure that you are including the correct filename in your html file. For example, in phone-dev.shtml, a file named jquery.js is included. The file you downloaded will likely have version numbers in it’s file name. So rename the downloaded file or change the content of src in your includes. This is uber trivial, but I’ve made the mistake several times.

You’ll also need JsSIP.js which can be downloaded here. Same naming care must be taken as is the case for jQuery and ArbiterJS. The recently added Instant Messaging component and some of the new features need jQuery-UI - so far version 1.11.* is known to work. From the downloaded .zip all you need is the jquery-ui-...js file and the jquery-ui-...css file, also to be placed in the project’s root directory. If you’ll be using the internationalization support you’ll also need jquery.i18n.properties.js.

To try out JSCommunicator, deploy the website locally by copying your project directory to the apache document root directory (provided you are using apache, which is a good idea.). You’ll likely have to restart your server before this works. Now the demo .shtml pages only have a header with all the necessary includes, then a Server Side Include for the body of the page, with all the JSCommunicator html. The html content is in the file jscommunicator.inc. You can enable SSI on apache, OR you can simply copy and paste the content of jscommunicator.inc into phone-dev.shmtl. Start up apache, open a browser and navigate to localhost/jscommunicator/phone-dev.shmtl and you should see:

Actually, pending updates to JSCommunicator, you should see a brand new UI! But all the core stuff will be the same.

Architecture

Disclaimer: I’m explaining my view of the JSCommunicator architecture, which currently may not be 100% correct. But so far it’s been enough for me to make my modifications and additions to the code, so it could be of use. One I get a JSCommunicator Founding Father’s stamp of approval, I’ll be sure to confirm the accuracy.

Now to explain how the JSCommunicator code interacts, the use of each code ‘item’ is described, ignoring all the html and css which will vary according to how you choose to use JSCommunicator. I’m also not going to explain jQuery which is a dependency but not specific to WebRTC. The core JSCommunicator code is the following:

config.js
jssip-helper.js
parseuri.js
webrtc-check.js
JSCommUI.js
JSCommManager.js
make-release
init.js
Arbiter.js
JsSIP.js

Each of these files will be presented in what I hope is an intuitive order.

config.js - As expected, this file contains your custom configuration specifications, i.e. the servers being used to direct traffic, authentication credentials, and a series of properties to enable/disable optional functionalities in JSCommunicator.

The next three files could be considered ‘utils’:

jssip-helper.js - This will load the data from config.js necessary to run JSCommunicator, such as configurations relating to servers, websockets, connection specifications (timeout, recovery intervals), and user credentials. Properties for optional features are ignored of course.
parseuri.js - Contains a function to segregate data from a URI.
webrtc-check.js - Verifies if browser is WebRTC compatible by checking if it’s possible to enable camera and microphone access.

These two are where the magic happens:

JSCommUI.js - Responsible for the UI component, controling what becomes visible to the user, audio effects, client-side error handling, and gathering the data that will be fed to JSCommManager.js.
JSCommManager.js - Initializes the SIP User Agent to manage the session including (but not limited to) beginning/terminating the session, sending/receiving SIP messages and calls, and signaling the state of the session and other important notifications such as incoming calls, messages received, etc.

Now for some extras:

make-release - Combines the main .js files into one big file. Gets jssip-helper.js, parseuri.js, webrtc-check.js, JSCommUI.js and JSCommManager.js and spits out a new file JSComm.js with all that content. Now you understand why phone-dev.shtml included each of the 5 files mentioned above whereas phone.shmtl includes only JSComm.js which didn’t exist until you ran make-release. That confused me a bit.
init.js - On page load, calls JSCommManager.js’ init function, which eventually calls JSCommUI.js’ init function. In other words, starts up the JSCommunicator app. I guess it’s only used to show how to start up the app. This could be done directly on the page you’re embedding JSCommunicator from. So maybe not entirely needed.

Third party code:

Arbiter.js - Javascript implmentation of the publish/subscribe patten, written by Matt Kruse. In JSCommunicator it’s used in JSCommManager.js to publish signals to describe direct the app’s behavior. For example, JSCommManager will publish a message indicating that the user received a call from a certain URI. In event-demo.js we subscribe to this kind of message and when said message is received, an action can be performed such as adding to the app’s call history. Very cool.
JsSIP.js - Implements the SIP WebSocket transport in Javascript. This ensures the transportantion of data is done in adherence to the WebSocket protocol. In JSCommManager.js we initialize a SIP User Agent based in the implementation in JsSIP.js. The User Agent will ‘translate’ all of the JSCommunicator actions into SIP WebSocket format. For example, when sending an IM, the JSCommunicator app will collect the essential data such as say the origin URI, destination URI and an IM message, while the User Agent is in charge of formating the data so that it can be transported in a SIP message unit. A SIP message contains a lot more information than just the sender, receiver and message. Of course, a lot of the info in a SIP message is irrelevant to the user and in JSCommUI.js we filter through all that data and only display what the user needs to see.

Here’s a diagram of sorts to help you visualize how the code interacts:

In sum, 1 - JSCommUI.js handles what is displayed in the UI and feeds data to JSCommManager.js; 2 - JSCommManager.js actually does stuff, feeding data to be displayed to JSCommUI.js; 3 - JSCommManager.js calls functions from the three ‘utils’, parseuri.js, webrtc-check.js and jssip-helpher.js which organizes the data from config.js; 4 - JSCommManager.js initializes a SIP User Agent based on the implementation in Arbiter.js.

When making any changes to JSCommunicator, you will likely only be working with JSCommUI.js and JSCommManager.js.

Kickstarting the JSCommunicator-xTuple Extension

Fri, 25 Jul 2014 15:06:52 +0000

xTuple is in my opinion incredibly well designed; the code is clean and the architecture ahderent to a standardized structure. All this makes working with xTuple software quite a breeze.

I wanted to integrate JSCommunicator into the web-based xTuple version. JSCommunicator is a SIP communication tool, so my first step was to create an extension for the SIP account data. Luckily for me, the xTuple development team published an awesome tutorial for writing an xTuple extension.

xTuple cleverly uses model based business objects for the various features available. This makes customizing xTuple very straightforward. I used the tutorial mentioned above for writing my extension, but soon noticed my goals were a little different. A SIP account has 3 data fields, these being the SIP URI, the account password and an optional display name. xTuple currently has a business object in the core code for a User Account and it would make a lot more sense to simply add my 3 fields to this existing business object rather than create another business object. The tutorial very clearly shows how to extend a business object with another business object, but not how to extend a business object with only new fields (not a whole new object).

Now maybe I’m just a whole lot slower than most people, but I had a ridiculously had time figuring this out. Mind you, this is because I’m slow, because the xTuple documentation and code is understandable and as self-explanatory as it gets. I think it just takes a bit to get used to. Either way, I thought this just might be useful to others so here is how I went about it.

Setup

First you’ll have to set up your xTuple development environment and fork the xtuple and xtuple-extesions repositories as shown in this handy tutorial. A footnote I’d like to add is please verify that your version of Vagrant (and anything else you install) is the one listed in the tutorial. I think I spent like two entire days or more on a wild goose (bug) chase trying to set up my environment when the cause of all the errors was that I somehow installed an older version of Vagrant - 1.5.4 instead of 1.6.3. Please don’t make the same mistake I did. Actually if for some reason you get the following error when you try using node:

<<ERROR 2014-07-10T23:52:46.948Z>> Unrecoverable exception. Cannot call method 'extend' of undefined

    at /home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:37:39

    at Object.<anonymous> (/home/vagrant/dev/xtuple/lib/backbone-x/source/model.js:1364:3)
    ...

chances are, you have the wrong version. That’s what happened to me. The Vagrant Virtual Development Environment automatically installs and configures everything you need, it’s ready to go. So if you find yourself installing and updating and apt-gets and etc, you probably did something wrong.

Coding

So by now we should have the Vagrant Virtual Development Environment set up and the web app up and running and accessible at localhost:8443. So far so good.

Disclaimer: You will note that much of this is similar - or rather, nearly identical - to xTuple’s tutorial but there are some small but important differences and a few observations I think might be useful. Other Disclaimer: I’m describing how I did it, which may or may not be ‘up to snuff’. Works for me though.

Schema

First let’s make a schema for the table we will create with the new custom fields. Be sure to create the correct directory stucture, aka /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>/database/source or in my case /path/to/xtuple-extensions/source/sip_account/database/source, and create the file create_sa_schema.sql, ‘sa’ is the name of my schema. This file will contain the following lines:

do $$
  /* Only create the schema if it hasn't been created already */
  var res, sql = "select schema_name from information_schema.schemata where schema_name = 'sa'",
  res = plv8.execute(sql);
  if (!res.length) {
    sql = "create schema sa; grant all on schema sa to group xtrole;"
    plv8.execute(sql);
  }
$$ language plv8;

Of course, feel free to replace ‘sa’ with your schema name of choice. All the code described here can be found in my xtuple-extensions fork, on the sip_ext branch.

Table

We’ll create a table containing your custom fields and a link to an existing table - the table for the existing business object you want to extend. If you’re wondering why make a whole new table for a few extra fields, here’s a good explanation, the case in question is adding fields to the Contact business object.

You need to first figure out what table you want to link to. This might not be uber easy. I think the best way to go about it is to look at the ORMs. The xTuple ORMs are a JSON mapping between the SQL tables and the object-oriented world above the database, they’re .json files found at path/to/xtuple/node_modules/xtuple/enyo-client/database/orm/models for the core business objects and at path/to/xtuplenyo-client/extensions/source/<EXTENSION NAME>/database/orm/models for exension business objects. I’ll give two examples. If you look at contact.json you will see that the Contact business object refers to the table “cntct”. Look for the “type”: “Contact” on the line above, so we know it’s the “Contact” business object. In my case, I wanted to extend the UserAccount and UserAccountRelation business objects, so check out user_account.json. The table listed for UserAccount is xt.usrinfo and the table listed for UserAccountRelation is xt.usrlite. A closer look at the sql files for these tables (usrinfo.sql and usrlite.sql) revealed that usrinfo is in fact a view and usrlite is ‘A light weight table of user information used to avoid punishingly heavy queries on the public usr view’. I chose to refer to xt.usrlite - that or I received error messages when trying the other table names.

Now I’ll make the file /path/to/xtuple-extensions/source/sip_account/database/source/usrlitesip.sql, to create a table with my custom fields plus the link to the urslite table. Don’t quote me on this, but I’m under the impression that this is the norm for naming the sql file joining tables: the name of the table you are referring to (‘usrlite’ in this case) and your extension’s name. Content of usrlitesip.sql:

select xt.create_table('usrlitesip', 'sa');

select xt.add_column('usrlitesip','usrlitesip_id', 'serial', 'primary key', 'sa');
select xt.add_column('usrlitesip','usrlitesip_usr_username', 'text', 'references xt.usrlite (usr_username)', 'sa');
select xt.add_column('usrlitesip','usrlitesip_uri', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_name', 'text', '', 'sa');
select xt.add_column('usrlitesip','usrlitesip_password', 'text', '', 'sa');

comment on table sa.usrlitesip is 'Joins User with SIP account';

Breaking it down, line 1 creates the table named ‘usrlitesip’ (no duh), line 2 is for the primary key (self-explanatory). You can then add any columns you like, just be sure to add one that references the table you want to link to. I checked usrlite.sql and saw the primary key is usr_username, be sure to use the primary key of the table you are referencing.

You can check what you made by executing the .sql files like so:

$ cd /path/to/xtuple-extensions/source/sip_account/database/source
$ psql -U admin -d dev -f create_sa_schema.sql
$ psql -U admin -d dev -f usrlitesip.sql

After which you will see the table with the columns you created if you enter:

$ psql -U admin -d dev -c "select * from sa.usrlitesip;"

Now create the file /path/to/xtuple-extensions/source/sip_account/database/source/manifest.js to put the files together and in the right order. It should contain:

{
  "name": "sip_account",
  "version": "1.4.1",
  "comment": "Sip Account extension",
  "loadOrder": 999,
  "dependencies": ["crm"],
  "databaseScripts": [
    "create_sa_schema.sql",
    "usrlitesip.sql",
    "register.sql"
  ]
}

I think the “name” has to be the same you named your extension directory as in /path/to/xtuple-extensions/source/<YOUR EXTENSION NAME>. I think the “comment” can be anything you like and you want your “loadOrder” to be high so it’s the last thing installed (as it’s an add on.) So far we are doing exactly what’s instructed in the xTuple tutorial. It’s repetitive, but I think you can never have too many examples to compare to. In “databaseScripts” you will list the two .sql files you just created for the schema and the table, plus another file to be made in the same directory named register.sql.

I’m not sure why you have to make the register.sql or even if you indeed have to. If you leave the file empty, there will be a build error, so put a ‘;’ in the register.sql or remove the line “register.sql” from manifest.js as I think for now we are good without it.

Now let’s update the database with our new extension:

$ cd /path/to/xtuple
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ psql -U admin -d dev -c "select * from xt.ext;"

That last command should display a table with a list of extensions; the ones already in xtuple like ‘crm’ and ‘billing’ and some others plus your new extension, in this case ‘sip_account’. When you run build_app.js you’ll probably see a message along the lines of “<Extension name> has no client code, not building client code” and that’s fine because yeah, we haven’t worked on the client code yet.

ORM

Here’s where things start getting different. So ORMs link your object to an SQL table. But we DON’T want to make a new business object, we want to extend an existing business object, so the ORM we will make will be a little different than the xTuple tutorial. Steve Hackbarth kindly explained this new business object/existing business object ORM concept here.

First we’ll create the directory /path/to/xtuple-extensions/source/sip_account/database/orm/ext, according to xTuple convention. ORMs for new business objects would be put in /path/to/xtuple-extensions/source/sip_account/database/orm/models. Now we’ll create the .json file /path/to/xtuple-extensions/source/sip_account/database/orm/ext/user_account.jscon for our ORM. Once again, don’t quote me on this, but I think the name of the file should be the name of the business object you are extending, as is done in the turorial example extending the Contact object. In our case, UserAccount is defined in user_account.json and that’s what I named my extension ORM too. Here’s what you should place in it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
[
  {
    "context": "sip_account",
    "nameSpace": "XM",
    "type": "UserAccount",
    "table": "sa.usrlitesip",
    "isExtension": true,
    "isChild": false,
    "comment": "Extended by Sip",
    "relations": [
      {
        "column": "usrlitesip_usr_username",
        "inverse": "username"
      }
    ],
    "properties": [
      {
        "name": "uri",
        "attr": {
          "type": "String",
          "column": "usrlitesip_uri",
          "isNaturalKey": true
        }
      },
      {
        "name": "displayName",
        "attr": {
          "type": "String",
          "column": "usrlitesip_name"
        }
      },
      {
        "name": "sipPassword",
        "attr": {
          "type": "String",
          "column": "usrlitesip_password"
        }
      }
    ],
    "isSystem": true
  },
  {
    "context": "sip_account",
    "nameSpace": "XM",
    "type": "UserAccountRelation",
    "table": "sa.usrlitesip",
    "isExtension": true,
    "isChild": false,
    "comment": "Extended by Sip",
    "relations": [
      {
        "column": "usrlitesip_usr_username",
        "inverse": "username"
      }
    ],
    "properties": [
      {
        "name": "uri",
        "attr": {
          "type": "String",
          "column": "usrlitesip_uri",
          "isNaturalKey": true
        }
      },
      {
        "name": "displayName",
        "attr": {
          "type": "String",
          "column": "usrlitesip_name"
        }
      },
      {
        "name": "sipPassword",
        "attr": {
          "type": "String",
          "column": "usrlitesip_password"
        }
      }
    ],
    "isSystem": true
  }
]

Note the “context” is my extension name, because the context + nameSpace + type combo has to be unique. We already have a UserAccount and UserAccountRelation object in the “XM” namespace in the “xtuple” context in the original user_account.json, now we will have a UserAccount and UserAccountRelation object in the “XM” namespace in the “sip_account” conext. What else is important? Note that “isExtension” is true on lines 7 and 47 and the “relations” item contains the “column” of the foreign key we referenced.

This is something you might want to verify: “column” (lines 12 and 52) is the name of the attribute on your table. When we made a reference to the primary key usr_usrname from the xt.usrlite table we named that column usrlitesip_usr_usrname. But the “inverse” is the attribute name associated with the original sql column in the original ORM. Did I lose you? I had a lot of trouble with this silly thing. In the original ORM that created a new UserAccount business object, the primary key attribute is named “username”, as can be seen here. That is what should be used for the “inverse” value. Not the sql column name (usr_username) but the object attribute name (username). I’m emphasizing this because I made that mistake and if I can spare you the pain I will.

If we rebuild our extension everything should come along nicely, but you won’t see any changes just yet in the web app because we haven’t created the client code.

Client

Create the directory /path/to/xtuple-extensions/source/sip_account/client which is where we’ll keep all the client code.

Extend Workspace View I want the fields I added to show up on the form to create a new User Account, so I need to extend the view for the User Account workspace. I’ll start by creating a directory /path/to/xtuple-extensions/source/sip_account/client/views and in it creating a file named ‘workspace.js’ containing this code:

XT.extensions.sip_account.initWorkspace = function () {

	var extensions = [
  	{kind: "onyx.GroupboxHeader", container: "mainGroup", content: "_sipAccount".loc()},
  	{kind: "XV.InputWidget", container: "mainGroup", attr: "uri" },
  	{kind: "XV.InputWidget", container: "mainGroup", attr: "displayName" },
  	{kind: "XV.InputWidget", container: "mainGroup", type:"password", attr: "sipPassword" }
	];

	XV.appendExtension("XV.UserAccountWorkspace", extensions);
};

So I’m initializing my workspace and creating an array of items to add (append) to view XV.UserAccountWorkspace. The first ‘item’ is this onyx.GroupboxHeader which is a pretty divider for my new form fields, the kind you find in the web app at Setup > User Accounts, like ‘Overview’. I have no idea what other options there are for container other than “mainGroup”, so let’s stick to that. I’ll explain content: “_sipAccount”.loc() in a bit. Next I created three input fields of the XV.InputWidget kind. This also confused me a bit as there are different kinds of input to be used, like dropdowns and checkboxes. The only advice I can give is snoop around the webapp, find an input you like and look up the corresponding workspace.js file to see what was used.

What we just did is (should be) enough for the new fields to show up on the User Account form. But before we see things change, we have to package the client. Create the file /path/to/xtuple-extensions/source/sip_account/client/views/package.js. This file is needed to ‘package’ groups of files and indicates the order the files should be loaded (for more on that, see this). For now, all the file will contain is:

enyo.depends(
"workspace.js"
);

You also need to package the ‘views’ directory containing workspace.js, so create the file Create the file /path/to/xtuple-extensions/source/sip_account/client/package.js and in it show that the directory ‘views’ and its contents must be part of the higher level package:

enyo.depends(
"views"
);

I like to think of it as a box full of smaller boxes.

This will sound terrible, but apparently you also need to create the file /path/to/xtuple-extensions/source/sip_account/client/core.js containing this line:

XT.extensions.icecream = {};

I don’t know why. As soon as I find out I’ll be sure to inform you.

As we’ve added a file to the client directory, be sure to update /path/to/xtuple-extensions/source/sip_account/client/package.js so it included the new file:

enyo.depends(
"core.js",
"views"
);

Translations

Remember “_sipAccount”.loc()” in our workspace.js file? xTuple has great internationalization support and it’s easy to use. Just create the directory and file /path/to/xtuple-extensions/source/sip_account/client/en/strings.js and in it put key-value pairs for labels and their translation, like this:

(function () {
  "use strict";

  var lang = XT.stringsFor("en_US", {
    "_sipAccount": "Sip Account",
    "_uri": "Sip URI",
    "_displayName": "Display Name",
    "_sipPassword": "Password"
  });

  if (typeof exports !== 'undefined') {
    exports.language = lang;
  }
}());

So far I included all the labels I used in my Sip Account form. If you write the wrong label (key) or forget to include a corresponding key-value pair in strings.js, xTuple will simply name your lable “_lableName”, underscore and all.

Now build your extension and start up the server:

$ cd /path/to/xtuple 
$ ./scripts/build_app.js -d dev -e ../xtuple-extensions/source/sip_account
$ node node-datasource/main.js

If the server is already running, just stop it and restart it to reflect your changes.

Now if you go to Setup > User Accounts and click the “+” button, you should see a nice little addition to the form with a ‘Sip Account’ divider and three new fields. Nice, eh?

Extend Parameters

Currently you can search your User Accounts list using any of the User Account fields. It would be nice to be able to search with the Sip account fields we added as well. To do that, let’s create the directory /path/to/xtuple-extensions/source/sip_account/client/widgets and there create the file parameter.js to extend XV.UserAccountListParameters. One again, you’ll have to look this up. In the xTuple code you’ll find the application’s parameter.js in /path/to/xtuple/enyo-client/application/source/widgets. Search for the business object you are extending (for example, XV.UserAccount) and look for some combination of the business object name and ‘Parameters’. If there’s more than one, try different ones. Not a very refined method, but it worked for me. Here’s the content of our parameter.js:

XT.extensions.sip_account.initParameterWidget = function () {

  var extensions = [
    {kind: "onyx.GroupboxHeader", content: "_sipAccount".loc()},
    {name: "uri", label: "_uri".loc(), attr: "uri", defaultKind: "XV.InputWidget"},
    {name: "displayName", label: "_displayName".loc(), attr: "displayName", defaultKind: "XV.InputWidget"}
  ];

  XV.appendExtension("XV.UserAccountListParameters", extensions);
};

Node that I didn’t include a search field for the password attribute for obvious reasons. Now once again, we package this new code addition by creating a /path/to/xtuple-extensions/source/sip_account/client/widgets/package.js file:

enyo.depends(
"parameter.js"
);

We also have to update /path/to/xtuple-extensions/source/sip_account/client/package.js:

enyo.depends(
"core.js",
"widgets"
"views"
);

Rebuild the extension (and restart the server) and go to Setup > User Accounts. Press the magnifying glass button on the upper left side of the screen and you’ll see many options for filtering the User Accounts, among them the SIP Uri and Display Name.

Extend List View

You might want your new fields to show up on the list of User Accounts. There’s a bit of an issue here because unlike what we did in workspace.js and parameter.js, we can’t append new things to the list of UserAccounts with the funciton XV.appendExtension(args). First I tried overwriting the original UserAccountList, which works but it’s far from ideal as this could result in a loss of data from the core implementation. After some discussion with the xTuple dev community, now there’s a better alternative:

Create the file /path/to/xtuple-extensions/source/sip_account/client/views/list.js and add the following:

1
2
3
4
5
6
7
8
9
var oldUserAccountListCreate = XV.UserAccountList.prototype.create;

XV.UserAccountList.prototype.create = function () {
  oldUserAccountListCreate.apply(this, arguments);
  this.createComponent(
  {kind: "XV.ListColumn", container: this.$.listItem, components: [
       {kind: "XV.ListAttr", attr: "uri"}
   ]})
};

To understand what I’m doing, check out the XV.UserAccountList implementation in /path/to/xtuple/enyo-client/application/source/views/list.js – the entire highlighted part. What we are doing is ‘extending’ XV.UserAccountList through ‘prototype-chaining’; this is how inheritance works with Enyo. In line 1 we create a prototype and in line 4 we inherit the features including original components array which the list is based on. We then create an additional component immitating the setup shown in XV.UserAccountList: An XV.ListColumn containing an XV.ListAttr, which should be placed in the XV.ListItem components array as is done with the existing columns (refer to implementation). Components can or should (?) have names which are used to access said components. You’d refer to a specific component by the this.$.componentName hash. The components in XV.UserAccountList don’t have names, so Enyo automatically names them (apparently) based on the kind name, for example something of the kind ListItem is named listItem. I found this at random after a lot of trial and error and it’s not a bullet proof solution. Can be bettered.

It’s strange because if you encapsulate that code with

XT.extensions.sip_account.initList = function () {
 //Code here
};

as is done with parameter.js and workspace.js (and in the xTuple tutorial you are supposed to do that with a new business object), it doesn’t work. I have no idea why. This might be ‘wrong’ or against xTuple coding norms; I will find out and update this post ASAP. But it does work this way. * shrugs *

That said, as we’ve created the list.js file, we need to ad it to our package by editing /path/to/xtuple-extensions/source/sip_account/client/views/package.js:

enyo.depends(
"list.js",
"workspace.js"
);

That’s all. Rebuild the app and restart your server and when you select Setup > User Accounts in the web app you should see the Sip URI displayed on the User Accounts that have the Sip Account data. Add a new User Account to try this out.

Contribute a JSCommunicator Translation

Thu, 17 Jul 2014 08:06:52 +0000

To those who don’t know, JSCommunicator is a SIP communication tool developed in HTML and JavaScript, currently supporting audio/video calls and SIP SIMPLE instant messaging. We’ve recently added i18n internationalization support for English, Portuguese, Spanish, French and now Hebrew. We would like to have support for other languages and it’s a pretty straightforward process to add a translation, so even if you do not have a tech background you can contribute!

Setup

First, create and account in Github. Github is a hosting service for git repositories and open source projects get to use Github for FREE! As a result, many open source project repositories are hosted on Github, including but not limited to JSCommunicator.

Once you have created your git account, follow steps 1-4 in the section Setting up git listed in Github’s setup tutorial. This will help you install git and configure anything you need.

Fork

Now fork the JSCommunicator repo. I really like the word ‘fork’. Great term. If this is your first fork, know that this will make a copy of the JSCommunicator repo and all its respective code to your Github account. Here’s how we go about it:

Sign in to Github
Browse to https://github.com/opentelecoms-org/jscommunicator.git
Somewhere around the upper left corner, you will see a button labeled ‘Fork’. Click that.

You should now have your own JSCommunicator repository on your Github account. On your main page select the tab ‘Repositories’. It should be the first on the list. The url will be https://github.com/YOUR-USERNAME/jscommunicator. My Github username is JLouback, so my jscommunicator repo can be found at https://github.com/JLouback/jscommunicator.

Clone

Now you should download all that code to your local machine so you can edit it. On your JSCommunicator page somewhere around the lower right corner you’ll see a field entitled HTTPS clone URL:

I don’t know if this is correct, but you can just copy the URL on your browser too. I do that.

Now on your terminal, navigate to the directory you’d like to place your repository and enter

git clone https://github.com/YOUR-USERNAME/jscommunicator

The current directory should now have a folder named ‘jscommunicator’ and in it is all the JSCommunicator code.

Switch branches

The JSCommunicator repo has a branch for the i18n support. A branch is basically a copy of the code that you can modify without changing the original code. It’s great for adding new features, once everything is done and tested you can add the new feature to the original code. A more detailed explanation of branches can be found here.

On the terminal, enter

cd jscommunicator
git branch -a

This will list all the branches in the repository, both local and remote. There should be a branch named ‘remotes/origin/i18n-support’. Switch to this branch by entering

git checkout --track origin/i18n-support

This will create a local branch named i18n-support with all the contents of the remote i18n-branch in https://github.com/opentelecoms-org/jscommunicator.

Create a .properties file

Now your jscommunicator directory will have some new files for the internationalization functionality. Among them you should find an INTERNATIONALIZATION_README file with instructions for adding a JSCommunicator translation. It’s a less detailed version of this post, but it’s worth a read in case I’ve missed something here.

Go to the directory ‘internationalization’. You’ll see a few .properties files with different language codes. Choose the language of your preference for the translation base and make a copy of it in the internationalization directory. Your copy should be named ‘Messages_LANGUAGECODE.properties’. For example, the language code for german is ‘de’, so the german file should be named ‘Messages_de.properties’. If you’re not sure about the code for your language, please check out this list of ISO 639-1 language codes so you’re using the same code as (most) browsers.

The .properties file is list of key-value pairs, a word or a few words joined by ‘_’ which is the key, the equals sign ‘=’ then a word or phrase which is the value. Do not change anything to the left of the equals sign. Translate what is on the right of the equals sign.

Modify available_languages.xml

Go back to the root directory (jscommunicator) and open the file named ‘available_languages.ruby’. This .ruby has a series of language elements. At the end of the last ‘</language>’ add a new language element with your language display name and code, copying the previous languge elements. You actually can put your language element anwhere, as long as it’s after the ‘' and before the , as well as not cutting into any of the other language elements. Your display name should be the name of the language in its native language, for example for a french translation we’ll put ‘Français’ (I think). And the code is the code we used for the .properties file. Here’s how it should look:

<language>
    <display>Deutsch</display>
    <code>de</code>
</language>

Save your changes and voila!

Test your work

Once you’ve made a .properties file, JSCommunicator will automatically load your translation if you’ve set your browser preference to that language. If your browser preference is french, it will load the Messages_fr.properties. If your browser preference is a language we don’t have a translation for (let’s say german), JScommunicator will load the default Messages.properties file which is in english.

The available_languages.ruby is used to build a language selection menu. If you add a language element without a corresponding .properties file, JSCommunicator will throw a JS error and load the default (english) translation. The same happens if you use different language codes in the .properties file name and the available_languages.ruby. The error won’t disturb your use of JSCommunicator, it just won’t load the language you selected. No harm, no foul. But if you want others to use your translation, do take care to do this correctly.

If you read the INTERNATIONALIZATION_README, you will have seen that we need the jquery.i18n.properties.js file that’s included in the .html pages. You can download that code here. Be sure to place it in the jscommunicator directory. This is only necessary if you want to see your addition at work, you don’t need this file to contribute your translation. There are other 3rd party code dependencies to run JSCommunicator too as you may have noticed. Just creating the .properties file and adding a language element to available_languages.ruby, if done correctly, is enough to make a pull request.

Commit

Here’s the part where we load all your local changes to your remote repository. In the terminal, navigate to the root directory (jscommunicator) and enter

git add internationalization/Messages_de.properties
git add available_languages.ruby

Of course, please replace ‘Messages_de.properties’ with the name of your new .properties file. And mind you, we don’t have a german translation yet!

Then enter

git commit -m "German translation"

You can enter whatever you’d like in the “ “ part, it’s a good idea to explain what language you are adding. Now we can push the changes:

git push -u origin i18n-support

You should now see the changes you made in your Github account page at github.com/YOUR_USER/jscommunicator. The message you put in double quotations (“ “) in the commit will be beside each modified file. This push also creates a new branch in your jscommunicator repository, now you will have a ‘master’ and a ‘i18n-support’ branch.

Pull

Now it’s time to share your translation with everyone else if you feel so inclined. Your version of JSCommunicator has a new translation but not the official version. First navigate to you jscommunicator repo at github.com/YOUR_USER/jscommunicator and make sure you are on the i18n-branch as that will contain your changes.

Click the green button directly to the left of the branch drop down menu shown above. This will create a pull request to add your new code to the official jscommunicator code. Once you click that button, you should see this:

Make sure that the base repo (the first on the line above the green Create pull request button) is opentelecoms-org:i18n-support and not opentelecoms-org:master or another branch name. If it is, just click ‘Edit’ on the right and you’ll be able to select the branch from a dropdown like so:

If first repo is opentelecoms-org:i18n-support and the second is YOUR_USER:i18n-support and you’ve verified (scroll down) that there are 2 files changed which are your new .properties file and the added element to the available_languages.xml, go ahead and press create pull request. It’ll open a window with a text box you can add an explanation to with one final ‘Create pull request’ button. Click that and you’re done! You’ve kindly contributed with a translation for JSCommunicator!