A brief summary of what has been written in the proposal As stated in the project proposal the problem statement is as follows finding a job as a student, many websites and their salaries are spread out. The process is as follows: graduation dates are approaching. You begin searching for the latest available internships on LinkedIn. You find a couple, but the specific criteria you're looking for don't match. You then start looking elsewhere for available internships, let's say Glassdoor, for instance. Still, you then face the same process: navigating through a web of jobs across the internet and comparing to your intended criteria.
Brief introduction of the project The purpose of this paper is to come up with a solution to the decentralized job search we, students, currently face today. The application will be pretty simple at face value, and we will centralize internship roles using already available data sets (Job-related websites). Under the hood, we'll be using specific tools to bring it all together, for example;
- You start with a list of potential jobs pulled from various websites as placeholders on the front of the page. • Additionally, you have the search function that searches for potential job roles.
- Once you've clicked on "Software development," you're then taken to a page where you'll see all of the software development roles with their respective salary pulled from different websites. a. This should be a list of roles with their respective salary.
- Lastly, the user can click on the role, and they are taken to the actual website of that role.
- Salary needs to be at the forefront of the UI itself.
Our application involves four distinct parts that are ultimately connected. First, we have our main PostgreSQL generated by Prisma (an ORM to create and access SQL databases). Second, our scraper app is written in Java. Third, we have our backend, which queries the database and exposes our JSON API. Finally, we serve everything to our front-end app written in ReactJS. Before showing anything to our front and backend, we must inject our database with some data. We do so by running our java application in the scraper-job folder, and this will prompt you to a couple of questions before you can execute the scraper. Here's how it works visually. This video showcases visiting one website searching for "React" and scrapping ten jobs. Here’s a link to the video. https://share.vidyard.com/watch/5kcyFeXHVzPukeSjfW6uUc?
Nice! Now that we've pulled ten jobs, we can later access them by using our API. Before we do that, though, we'll need to get more jobs for our application. We'll go ahead and rerun our app, but this time we'll get multiple select websites to pull our jobs from (we use threads under the hood to remove various positions from different websites). Here's how it works visually, we search for "React" this time on all the websites we haven't pulled from. https://share.vidyard.com/watch/NZKz47CDfxz9gKwj56ViMN?
Now that we have injected our database with some data, we'll need to view it in our database. We can do so by running our Prisma client to visualize our database!
In our backend we create multiple queries for our application like, search, getAllJobs, and getJobTypes. As you can see below each query will look like so in our express app. This query will get each job type. We also call extractDuplicate, to extract all of the duplicates from our query since every job type will have the following, React, Product manager, or Backend Engineer.
Search Our search function gives us the ability to look for a specific job title. Here's how've we implemented our search query in the backend there different layers to it.