Janice Darikho


Indonesian UX designer and developer, Singapore-based

Libri: CLI Scraper RubyGem for Bibliophiles

I’ll share my process-based approach on how I created Libri and published it on RubyGems.org, alongside some technical roadblocks that I faced during the development phase. This project specifically focuses on scraping, which is a term used to describe the act of retrieving HTML-and-CSS-based data from a website page. Here is a walkthrough video of how Libri works:

1. Discovery

After sifting through several scraping ideas—including scraping Noti.st, or 80,000 Hours’ Problem Profiles, or Adafruit’s Raspberry Pi projects—I settled on going back to a theme that can be simple, meaningful, and usable by many: Books. In searching for which website to scrape from, I had several options: the Man Booker website, Goodreads’ Awards section, as well as Penguin’s Award Winners list.

I chose Barnes & Noble’s awards webpage to scrape as it seems to be the most comprehensive and it’s also quite up-to-date.

2. Strategy

To build a gem using Bundler, I started by running bundle gem libri in the terminal at the Libri working directory. This will create file structures (called scaffold directory) for our gem, so we can start coding right away.

I made sure that my computer has also installed the following dependencies:

  • Rake, used to build a local copy of our gem, which we’ll use to push and publish to RubyGems.org
  • OpenURI, used to open a URL as if it is an HTML file
  • Nokogiri, used to parse HTML and XML values from a webpage
  • Pry, used as a local sandbox and a debugging tool
  • Colorize, used to style text in the terminal using different colours

3. Architecture

Now, for Libri, I wanted to make 3 things work on my terminal:

  • Display the various awards
  • Display the books belonging to a chosen award
  • Display the information of a chosen book

To do this, I structured my lib folder in this manner, separating the CLI, scraper, awards, books, and book classes.

Simplified directory structure for Libri

Each class is responsible for different parts of the gem:

  • The CLI class is responsible for the terminal interface that interacts with the user
  • The scraper class scrapes text-based contents off the webpage
  • The awards class creates new instances of Awards object from hash values returned by the Scraper.scrape_barnes_noble method
  • The books class creates new instances of Books object from hash values returned by the Scraper.scrape_award(award) method
  • The book class creates new instances of Books object from hash values returned by the Scraper.scrape_book(book) method

4. Development

This stage took the longest to complete, but all in all, it was a success, and I have several notes to make:

  • I learned to use a multi-line string via HEREDOC, which in itself has various methods to achieve the same thing (e.g. %{...}, %Q{...}, <<-EOS...EOS)
  • Initially, whenever an exit command was called, the Please try again. message would also be displayed. This was fixed by using a single-level if/else...end conditional rather than while input != 'exit'...end loop.
  • I knew that I wanted to access several levels of information, scraping from various URLs, and being able to pass in different URL based on the user’s input (e.g. if user inputs for the Pulitzer Awards, the Scraper.scrape_award() method must return information based on the Pulitzer Awards URL. If user inputs for Man Booker Prize, the expected return should be from the Man Booker URL). I knew then that I needed to pass in the URL as an argument for the Scraper.scrape_award() method. Knowing this, I included a :url key into the top-level awards hash, whose value will be passed in to Scraper.scrape_award(). Then, the second-level books hash can scrape from and access from the passed in URL—the same concept applies as we scrape from a third-level URL for individual book information. I wasn’t sure if this was workable, as previous labs I worked on hasn’t used a multi-level, real-time updated website, and therefore had no need for this flow. But it was! This was the best revelation I learned while building this project, knowing that versatility can be built into code.
  • I couldn’t access HTML values for attributes which are not href. The rating values on the B&N website was stored within the aria-label attribute, which does not return a value when I attempted to access it. I also couldn’t access the books listed under the Customers Who Bought This Item Also Bought section, which returns nothing as well. I’m still searching for answers.
  • Originally, upon scraping, I realised that I could access hash values and display them from the CLI class using Hash[:key], even without instantiating new objects and assigning them their arguments / attributes. This led to an oversight, where I published the working gem without practicing the Ruby object relationship methods, such as has-many. This was fixed by editing the awards, books, and book classes accordingly. Now we can access hash values, such as book.title and book.author using the attr_accessor.
  • At one point, as the terminal displayed a list of books, then went back to select another award, the list of books displayed was accumulated, resulting in 20–40–60… number of books. This was a disaster, and I had almost given up. However, it was soon realised that the bug was caused by CLI#make_book(award) method being called every time CLI#menu_award was called, and this adds a new array of books onto Books.all. #make_book(award) is needed to instantiate our Book object and to access various attributes of our Book, and we need that. To fix this, a method to clear the previous instantiated object was included before #make_book(award) is called, thus resetting the Books.all return value for each menu call.

All in all, I wouldn’t have been able to overcome these challenges without talking through my code line by line, component by component, flow by flow, as suggested by Dakota.

By talking out my thought process based on this rough flow:

  • What am I trying to do?
  • Is Ruby doing what I’m expecting her to do? (Yn)
  • If no, what’s happening instead, and why do we think it’s happening?
  • If it’s happening because of X—then, if we change Y, we expect Z to happen.
  • We test our hypothesis by changing Y, and we see if Z happens.
  • If Z happens, based on our understanding of X, we should know how to fix it and achieve what we were trying to do.
  • If Z doesn’t happen, don’t give up! Read up and look for help, and test different understandings to find the one that works with Ruby.
  • This is a simple project, however, with several different components interacting with one another, it was soon quite easy to lose track of one of them (e.g. how best to access and display every single piece of information, at which stage have objects been instantiated and at which stage they have not been, etc.), and when I lost that one, I soon lost focus on the big picture and I had to start all over again. So here’s to remember to keep practicing, and to practice it right!

5. Publishing

Lastly, to publish a gem for the first time, I followed these simple steps:

  1. Edit the Gemspec file and update the Summary as well as Description specification. Make sure that all todo on the file has been rewritten to prevent any potential errors when publishing. Next, comment out the entire code block that says Prevent pushing this gem to RubyGems.org, otherwise we won’t be able to push our gem.
  2. Change spec.bindir and spec.executables.
  3. Add dependencies via spec.add_development_dependency and spec.add_dependency.
  4. Update the version.rb file if necessary, following semantic versioning standards. There are many guides out there, including this and this.
  5. Update the README.md file as well. This is to help users have an overview of the gem, as well as how to install and run the gem.
  6. Make sure that your GitHub repo has all its files updated (latest commit and push).
  7. Make sure that rake is installed—so we can run rake build followed by rake release, which will push our latest gem version onto RubyGems.org for others to use! Alternatively, I also tried using gem build and gem push libri-0.x.x.gem to a similar effect. Another alternative is to install the gem-release gem, which provides several methods for helping with gem development that I will explore with further projects.

Hope you enjoyed this post, and I hope that makes sense to you! Drop in any suggestions for the gem and I’ll work on it. Happy coding! Originally published on Medium here.


Learning Object Relationships in Ruby with Pokémons

After spending the better part of my Sunday trying to understand Object Relationship, and still not getting anywhere near a lightbulb moment (not even a dim one), I travelled across the Internet land for the very best explanation, searching far and wide.

That’s when I stumbled upon this short post by Han Lee, where he described object relationship using Pokémon. It was so good! But so short! 🤕

So I decided to take it further. In essence, object relationship is a concept used to illustrate how different instances of our classes can interact with one another, as with real-life situations. Our class instances can also be referred to as models. There are several basic ways in which models relate to one another—in this case, I’m going to practice the belongs to and the has many relationships.

In the world of Pokémons, you can have many Pokémons (e.g. Pikachu, Eevee, etc.). In a reciprocal manner, those Pokémons belong to you. And because you love Pokémons and you want to teach them more, each of your Pokémon also has many moves and those moves belong to each Pokémon with a specific type (e.g. electric, grass, etc.).

So let’s set up our three separate classes: Trainer, Pokemon, and Move.

As we commit to become the very best trainer, like no one ever was, Prof. Oak called us to his lab and asked for our trainer_name. At this point, we still have no Pokémons and our array starts empty. But no worries! We’ll earn our first starter Pokémon in no time!

class Trainer
  attr_accessor :trainer_name, :pokemons

  def initialize(trainer_name)
    @trainer_name = trainer_name
    @pokemons = []
  end
end

Alright. Now, let’s set up our Pokémons! Since our Pokémons belong to us, we are assigning attr_accessor :trainer at the beginning of the class. Next, our class Pokemon is responsible for recording all the Pokémons that we are going to encounter, and push them into our global variable @@pokedex. When we encounter a new Pokémon, we also need to record its pokemon_name and pokemon_type, among other things, but we’ll keep it to these two for simplicity! And because we’re still a Lv 1 trainer, most of our Pokémons have not been taught any special moves yet, and our instance variable pokemon_moves initiates to an empty array.

class Pokemon
  attr_accessor :pokemon_name, :pokemon_type, :pokemon_moves, :trainer
  
    @@pokedex = []

  def initialize(pokemon_name, pokemon_type)
    @pokemon_name = pokemon_name
    @pokemon_type = pokemon_type
    @@pokedex << self
    @pokemon_moves = []
  end
end

We’re also going to create a Move class, not to complicate things, but because we promised to teach our best friends a few chops and kicks! Since we can only teach certain moves to Pokémons of a corresponding type, we also need to define pokemon_type when we’re creating a new instance variable move.

class Move
  attr_accessor :pokemon_type, :move

  def initialize(pokemon_type, move)
    @pokemon_type = pokemon_type
    @move = move
  end
end

Now, we’re done setting up! Let’s go back to our Trainer class. We’ve chosen Pikachu as our first starter, and we need a method within our class to add Pikachu onto our slots. Our #add_pokemon method will, well, add our Pokémon for us. And by assigning self to our pokemon.trainer, we automatically reciprocates the relationship between new Pokémon and ourselves, the trainer. Lastly, we can also check each Pokémon in our current slots using the #pokemon_slots method, which will also helpfully tell us our Pokémon’s type.

class Trainer
  attr_accessor :trainer_name, :pokemons

  def initialize(trainer_name)
    @trainer_name = trainer_name
    @pokemons = []
  end

  def add_pokemon(pokemon)
    @pokemons << pokemon
    pokemon.trainer = self        # The added Pokemon belongs to the trainer whom we called #add_pokemon on
  end
  
  def pokemon_slots
    @pokemons.map { |pokemon|
      "#{pokemon.pokemon_name} : #{pokemon.pokemon_type}"
    }
  end
end

So, let’s check on our code by calling on our methods!

ash = Trainer.new("Ash")
pikachu = Pokemon.new("Pikachu", "electric")
bulbasaur = Pokemon.new("Bulbasaur", "grass")
blastoise = Pokemon.new("Blastoise", "water")

ash.add_pokemon(pikachu)
ash.add_pokemon(bulbasaur)
ash.add_pokemon(blastoise)

ash.pokemon_slots
=> ["Pikachu: electric", "Bulbasaur: grass", "Blastoise: water"]

Now, next, after levelling up some, we want to teach our friends some sick moves. How do we do that? We already have our Move class, which we can use to call new moves anytime. What’s next? We probably need to define some methods in our Pokemon class so that we can assign a new move to a Pokémon. But our Pokémons can only learn moves that correspond to their types, so we need to code that conditional in as well. And finally, to look at all the moves each Pokémon knows, we use the known_moves method.

class Pokemon
  attr_accessor :pokemon_name, :pokemon_type, :pokemon_moves, :trainer

    @@pokedex = []

  def initialize(pokemon_name, pokemon_type)
    @pokemon_name = pokemon_name
    @pokemon_type = pokemon_type
    @@pokedex << self
    @pokemon_moves = []
  end

  def assign_move(move)
    if self.pokemon_type == move.pokemon_type
      puts "#{pokemon_name} learned #{move.move}!"
      @pokemon_moves << move
      move.pokemon << self
    else
      puts "#{pokemon_name} can't learn #{move.move}!"
    end
  end

  def known_moves
    @pokemon_moves.map { |move|
      move.move
    }
  end
end

Let’s try calling our new methods!

pikachu.assign_move(thunderbolt)      # =>  "Pikachu learned Thunderbolt!"
pikachu.assign_move(surf)             # =>  "Pikachu can't learn Surf!"
pikachu.assign_move(frenzy_plant)     # =>  "Pikachu can't learn Frenzy Plant!"
pikachu.assign_move(catastropika)     # =>  "Pikachu learned Catastropika!"

pikachu.known_moves
=> ["Thunderbolt", "Catastropika"]

All done! Here, we learn about how each class can use the informations in other classes to its advantage, especially with the help of attr_accessors. We also learned about how to access those information and represent them as strings or array, using Ruby’s built-in map method. We also used self whenever we want to use objects (data and behaviour) to describe something. We got to practice some conditionals along the way, too. So that’s it! Hope that was as much fun for you as it was for me!


All flub-ups and boo-boos, if there are any, are mine. Any questions, shoot a message anytime! I don’t bite. Follow me on Twitter at @jouissances. Originally published on Medium here.


First Step

This post marks my first step towards becoming a career Full Stack developer.

I’m not, by any means, the first mid-career changer who decided to pursue tech as a lifetime career. For me, that original career path is as a medical representative in Singapore, and eventually becoming a medical science liaison (MSL).

I first dabbled in code when I was 16, designing custom HTML and CSS post templates for online Harry Potter RPG forums. I didn’t know it was called ‘code’ then, I just figured it made things look and work better. One thing led to another, and after my college graduation, I had plenty of free time while job-searching, and I clicked on an ad that led me to Codecademy. I spent many hours learning the basics of Front End, and after a week, I decided to enrol in one of the Pro Intensive courses. As of now, they have 9 Intensive courses, from Build Websites from Scratch, Build Front-End Web Apps, Build Website UIs, Programming with Python, to Data Analysis. I completed the first three courses, but I still didn’t feel satisfied. I would browse through SiteInspire and I would get a headache while trying to apply my current skills to build those products. I couldn’t. I wanted to learn so much more. By now, 4 months have passed and I was just starting with my current job.

Prior to this job, I have never faced so many rejections in a day. Nevertheless, it taught me about perseverance and gradually, I learned a few things about the trade and how I can improve. I still learn to code after work hours, putting in hour after hour to this newfound hobby. I started asking my clients if they would like a freelance web development service, and several said yes! I’ve never been so excited — finally, I wasn’t offering a product that others made, I was offering my own product. Something that I can call my own and be proud of, something that potentially can help users and benefit the clients.

During these times, I sought out help like no other — I signed up for more courses, including design (and eventually, UI/UX Design), parsed through many arrays of resources (which I’ve collated here) and determined which ones are worth the time and cost. I practiced a lot harder. I made CodeNewbie and User Defenders part of my daily commute routine. I joined local coding Slack channels, attended free meetups, and approached potential mentors who I really admire and respect. I have never felt more exhausted, but at the same time, I have never felt so fulfilled, motivated, and thrilled about something. After mulling for a couple of weeks, it was Saron’s confidence, encouragement, and prowess that finally convinced me to join Flatiron School. And after completing the Bootcamp Prep, I enrolled with a Women Take Tech scholarship (made possible by Flatiron and Lyft) under my belt.

There are many reasons why I believe that tech is a thriving industry and always will be. I’ll share a few here:

  1. The possibility of creating and inventing any tech product is almost limitless.
    There isn’t an industry quite like this. The improvements in AI (think Google Duplex), science, and software languages in the past decade alone is enough proof that technology can be unstoppable. Furthermore, depending on the project scope, the cost of building a product can virtually be zero — except for time and effort.

  2. The intersection of many different disciplines with tech.
    The typical pillars of industrial sectors include STEM, arts, humanities, education, business, finance, and security. Tech is one sector which is able to connect multiple disciplines, allowing a diversity of resources to interact and create something unique. I’d like to think that in the future, these different areas would have integrated tech in their system so deep, that everything will become dependent on tech to operate efficiently, and thus increasing the demand for skilled engineers.

  3. The community is superb, supportive, and incredibly resourceful.
    There is something about the tech community that is truly encouraging. From open-source projects to a casual extended helping hand from a senior developer, there is no place where a newbie developer wouldn’t feel welcome. Some developers can be quite critical, yes, but there are so many others who are willing to help and guide a newbie along on the right path. There is no metropolitan city on Earth that doesn’t have a local coding community meetup, or an annual conference, or a coding bootcamp. Also, #MINSWAN.

  4. Coding allows for the expression of both an individual’s creative and logical processes.
    Firstly, this may not directly contribute to the progress of tech as a whole, and secondly, it is a lesser known concept about programming, which is substantiated here. As a discipline, computer science and data theories may seem to have a tendency to focus more towards logical thinking; however, the fact that there are many ways to solve a problem proves that creativity is required in the field of programming. And at an individual’s level, this feature of coding can be very liberating, encouraging more and more learners to participate in the ecosystem.

I have many other slew of reasons, but I’m sure that I might be preaching to the choir here. I’m also very opinionated about the importance of design in creating impactful products, which is why I’m also enrolled in a UX Design course. After my Flatiron graduation, I would like to find a Junior Developer position where I’m also allowed to be flexible and participate in the UX research team, or vice versa. I would like to find a job that provides me with a much better financial resource, future prospect, and autonomy when it comes to learning continuously. I would love to find a job that allows me the freedom to work remotely, preferably with a small, capable, and ambitious team.

In 5 years’ time, I hope to be able to also build a remote digital agency that focuses on creating impact for SMEs and non-profits.

There are many things I’m not confident about — such as the hiring climate in Singapore by the time I graduate, the discrepancy between the advertised salaries on bootcamps and the actual salaries which local companies are willing to offer. I would even be lying if I was to say that I’m perfectly confident that everything will go easy during this bootcamp, especially with the GMT +8 timezone and 3AM study groups.

But here’s to never giving up.