Angela Petteys, Author at ProdSens.live https://prodsens.live/author/angela-petteys/ News for Project Managers - PMI Tue, 14 May 2024 19:20:20 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 https://prodsens.live/wp-content/uploads/2022/09/prod.png Angela Petteys, Author at ProdSens.live https://prodsens.live/author/angela-petteys/ 32 32 Mentor Matching — May 2024 🤝 https://prodsens.live/2024/05/14/mentor-matching-may-2024-%f0%9f%a4%9d/?utm_source=rss&utm_medium=rss&utm_campaign=mentor-matching-may-2024-%25f0%259f%25a4%259d https://prodsens.live/2024/05/14/mentor-matching-may-2024-%f0%9f%a4%9d/#respond Tue, 14 May 2024 19:20:20 +0000 https://prodsens.live/2024/05/14/mentor-matching-may-2024-%f0%9f%a4%9d/ mentor-matching-—-may-2024-

This monthly series is dedicated to helping mentors and mentees connect with one another. 🤜💥🤛 The DEV Community…

The post Mentor Matching — May 2024 🤝 appeared first on ProdSens.live.

]]>
mentor-matching-—-may-2024-

This monthly series is dedicated to helping mentors and mentees connect with one another. 🤜💥🤛

The DEV Community is made up of a wide variety of developers who are at different points in their careers with separate skill sets and specialties. We’re constantly sharing our experiences and learning together in public, so it feels natural to open up a space for mentors and potential mentees to meet & mingle, discuss what they’re offering or hoping to get from mentorship, and plan their next steps.

Here’s How It Works

Below, we’ll offer some guidance for how to approach this thread from both a mentor’s and a mentee’s perspective.

Guidance for Mentors 🍎

If you’re a mentor and plan on using this thread to talk about your experience and what you can provide mentorship-wise, you’ve come to the right place. Also, thank you… we appreciate you offering to help others in our community!

NOTE: If you’re offering a paid-for service, we ask that you please be upfront about that in your message.

We recommend sharing a message about yourself that includes the following:

  1. Your experience as a developer and as a mentor. If it’s your first time offering mentorship, that’s okay; please just be honest.
  2. What you’re able to offer. For example: share your preferred method of communication and how much estimated time you can provide on a daily/weekly/monthly basis.
  3. Why you’re doing this. Perhaps you remember being a beginner and would like to help someone out. Maybe you run a supportive Discord channel. Or, maybe it’s part of your career in DevRel.
  4. How folks can connect with you. You could provide contact details, point to your website, link to your socials/Discord channel/etc. — just give folks a way to get in touch.

Guidance for Mentees ✨

Thanks for coming here to find a mentor and/or to put yourself out there as a potential mentee. We hope that this thread helps to connect you with somebody or group of folks in the DEV Community who can offer you guidance and assistance via mentorship.

NOTE: We do not endorse every person who lists themselves as a mentor in this thread. Remember, this is an open thread, so please be cautious about sharing your personal details. If you see someone whose behavior or services seem suspicious, please don’t hesitate to report them to us.

We recommend responding to a mentor who has let a message below and/or sharing a message about yourself that includes the following:

  1. Your current experience level as a developer. If you’re just beginning and/or in early learning stages, that’s absolutely okay!
  2. What your personal goals are. These can be long-term and aspirational, e.g. I want to become a professional web developer. They can also be more short-term and specific, e.g. I want to learn best practices for naming models in Ruby.
  3. What your personal commitment looks like. Can you meet up multiple times a week, once a week, once a month? Are you full-time job searching or currently working somewhere else? How much time do you expect to spend on this?
  4. How folks can connect with you. You could let a mentor know that you plan on contacting them. And/or you could provide contact details, point to your website, socials, etc.

Match on 🤝

A quick clip from the original Star Wars, Episode IV. A close up shot of Obi Wan Kenobi saying "You must learn the ways of the force"

Alrighty, now that y’all know how this works, feel free to drop your details below and let folks get to know you!

The post Mentor Matching — May 2024 🤝 appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/05/14/mentor-matching-may-2024-%f0%9f%a4%9d/feed/ 0
[ML Story] Fine-tune Vision Language Model on custom dataset https://prodsens.live/2024/05/08/ml-story-fine-tune-vision-language-model-on-custom-dataset/?utm_source=rss&utm_medium=rss&utm_campaign=ml-story-fine-tune-vision-language-model-on-custom-dataset https://prodsens.live/2024/05/08/ml-story-fine-tune-vision-language-model-on-custom-dataset/#respond Wed, 08 May 2024 17:20:00 +0000 https://prodsens.live/2024/05/08/ml-story-fine-tune-vision-language-model-on-custom-dataset/ [ml-story]-fine-tune-vision-language-model-on-custom-dataset

Introduction We’re living in the era of LLMs, and almost every week, you’ll hear a new language model…

The post [ML Story] Fine-tune Vision Language Model on custom dataset appeared first on ProdSens.live.

]]>
[ml-story]-fine-tune-vision-language-model-on-custom-dataset

Introduction

We’re living in the era of LLMs, and almost every week, you’ll hear a new language model making its way out. From Google’s Gemini and Gemma models to Meta’s latest Llama 3 and Microsoft’s tiny Phi-3 models, a fierce online competition is underway among these industry giants to grab the top spot.

Amidst this flurry of activity, what stands out most is the willingness of these tech giants to open up some of these language models to the developer community.

What are the benefits of making models open?
Opening up models to the developer community brings several advantages, including developers fine-tuning these models for specific use cases to solve interesting problems.

And if you’re an LLM nerd by now, I’m sure you have attempted to fine-tune at least one of these open models to explore their capabilities. While discussing all these popular models, it’s uncommon to find ones that are both open and multimodal simultaneously.

One such hidden gem I recently explored was the Idefics2-8B Vision Language Model built by Hugging Face 🤗. It’s open, and supports multi-modality, accepting sequences of image and text inputs. The Idefics2 model can answer questions about images, describe visual content, creating stories from multiple images, and a lot more.

I had been searching for articles and tutorials that detail the steps for fine-tuning Vision Language Models on custom datasets. While most covered the fine-tuning process (on already available datasets), they often overlooked the crucial step of data preparation.

This blog just brings that to you: A comprehensive guide to fine-tuning the Idefics2 model where you’ll not only learn how to fine-tune a Vision Language Model, but also preparing your own custom dataset from scratch.

But before we start, let’s quickly understand the high-level architecture of Vision Language Models, and how do they differ from standard LLMs.

Vision Language Models

Vision Language Models

Vision Language Models are multimodal models that learn from images and text, generating text outputs from image and text inputs. They excel in zero-shot capabilities, generalization, and various tasks like image recognition, question answering, and document understanding.

These models can also capture spatial properties and output bounding boxes or segmentation masks for specific subjects. Learn more about VLMs and their architecture here.

Now that you’ve a basic understanding of VLMs, it’s show time. Let’s dive into the code.

Step 1: Data Preparation

Unlike LLMs, the dataset format for VLMs is slightly different due to the introduction of images and videos in addition to the standard textual data.

Today, we’ll fine-tune the Idefics2 model on images of documents for visual question and answering. Our training data is derived from a sub-sampled version of the DocVQA dataset, with slight modifications to recreate the entire dataset from scratch.

Clone the following repository.

!git clone https://github.com/NSTiwari/Fine-tune-IDEFICS-Vision-Language-Model

Within the repository, you’ll find a datasets folder that has an images sub-folder. This sub-folder contains both the training and testing sets of images. Precisely, there are 1000 images for training and 200 images for testing.

Here’s a gist of how the data looks like:

Sub-sampled DocVQA image dataset

The images are black and white scanned copies of various documents like news articles, emails, invoices, reports, advertisements, and more.

In addition to the images, there’s a qa_text.csv file in the repository that contains details about all the images. Since we’re working on a question and answering task, the CSV file contains the questions or queries and their corresponding answers for each image.

Sub-sampled DocVQA CSV dataset

Now that we have both the images and their associated content stored separately, let’s merge them to create a dataset formatted for training the model.

Install the following libraries:

!pip install -q git+https://github.com/huggingface/transformers.gi 
!pip install -q accelerate datasets peft bitsandbytes
from datasets import Dataset, DatasetDict, Image
import pandas as pd
import os

# Define train and test size.
TRAIN_SAMPLES = 1000
TEST_SAMPLES = 200
TEST_SIZE = 0.166 #

# Define the directory containing the images.
train_images_directory = '/content/Fine-tune-IDEFICS-Vision-Language-Model/dataset/images/train/'
test_images_directory = '/content/Fine-tune-IDEFICS-Vision-Language-Model/dataset/images/test/'

# Read the CSV Q&A text.
qa_text = pd.read_csv('/content/Fine-tune-IDEFICS-Vision-Language-Model/dataset/qa_text.csv')

# Create a list of image paths.
train_image_paths = [os.path.join(train_images_directory, f'train_{i}.jpg') for i in range(TRAIN_SAMPLES)]
test_image_paths = [os.path.join(test_images_directory, f'test_{i}.jpg') for i in range(TEST_SAMPLES)]
image_paths = train_image_paths + test_image_paths

# Create a list of other columns such as id, query, and answer.
ids = ids = qa_text['id'].tolist()
queries = qa_text['query'].tolist()
answers = qa_text['answers'].tolist()

# Create the dataset dictionary.
dataset_dict = {
'id': ids,
'image': image_paths,
'query': queries,
'answers': answers
}

# Create the dataset.
dataset = Dataset.from_dict(dataset_dict)

# Cast the 'image' column to Image type.
dataset = dataset.cast_column("image", Image())

# Split the dataset into train and test.
split_dataset = dataset.train_test_split(test_size=TEST_SIZE, shuffle=False)

# Push the dataset on Hugging Face Hub.
split_dataset.push_to_hub("NSTiwari/DocumentIDEFICS_QA")

The above script combines the images with the textual queries and answers to create a unified dataset, which is subsequently uploaded to Hugging Face 🤗. You can access the dataset through this link.

DocumentIDEFICS_QA dataset on Hugging Face

Well done on creating your custom dataset. It was indeed straightforward, wasn’t it? If you wish to create a dataset for another use-case, simply follow the same steps as before.

Step 2: Load Dataset

Now that we have the dataset prepared, let’s proceed to load it.

from datasets import load_dataset

train_dataset = load_dataset("NSTiwari/DocumentIDEFICS_QA", split="train")
eval_dataset = load_dataset("NSTiwari/DocumentIDEFICS_QA", split="test")

Inspect the train data.

print(train_dataset[0])
train_dataset[0]['image']

This is how a record in the training dataset appears. The image is embedded as a JpegImageFile within the dataset.

{
'id': 'train_0',
'image': ,
'query': 'what is the date mentioned in this letter?',
'answers': "['1/8/93']"
}
train_0.jpg

Step 3: Configure LoRA adapters

Training or fine-tuning a language model is a herculean task in itself and is practically impossible unless you’re an owner of high computing GPUs.

With the introduction of Parameter-Efficient Fine-Tuning (PEFT), there’s no need to stress about training the entire language model anymore.

Thanks to techniques like LoRA and QLoRA, you can efficiently fine-tune these large models by significantly reducing the number of trainable parameters. This not only accelerates the fine-tuning process but also conserves memory usage.

import torch
from peft import LoraConfig
from transformers import AutoProcessor, BitsAndBytesConfig, Idefics2ForConditionalGeneration

DEVICE = "cuda:0"
USE_LORA = False
USE_QLORA = True

processor = AutoProcessor.from_pretrained(
"HuggingFaceM4/idefics2-8b",
do_image_splitting=False
)
if USE_QLORA or USE_LORA:
lora_config = LoraConfig(
r=8,
lora_alpha=8,
lora_dropout=0.1,
target_modules='.*(text_model|modality_projection|perceiver_resampler).*(down_proj|gate_proj|up_proj|k_proj|q_proj|v_proj|o_proj).*$',
use_dora=False if USE_QLORA else True,
init_lora_weights="gaussian"
)
if USE_QLORA:
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16
)
model = Idefics2ForConditionalGeneration.from_pretrained(
"HuggingFaceM4/idefics2-8b",
torch_dtype=torch.float16,
quantization_config=bnb_config if USE_QLORA else None,
)
model.add_adapter(lora_config)
model.enable_adapters()
else:
model = Idefics2ForConditionalGeneration.from_pretrained(
"HuggingFaceM4/idefics2-8b",
torch_dtype=torch.float16,
_attn_implementation="flash_attention_2", # Need GPUs like A100 or H100.
).to(DEVICE)

The above chunk of code configures the LoRA adapters for the Idefics2–8b model.

QLoRA, short for Quantized LoRA, is an enhanced iteration of LoRA. As the name suggests, QLoRA quantizes the precision of weight parameters, compressing a 32-bit parameter into a 4-bit format.

This sharp reduction in memory demands for the LLM enables easy fine-tuning, particularly beneficial when hardware resources are limited.

Step 4: Create Data Collator

Data collators are objects that will form a batch by using a list of dataset elements as input. These elements are of the same type as the elements of train_dataset or eval_dataset.

import random

class MyDataCollator:
def __init__(self, processor):
self.processor = processor
self.image_token_id = processor.tokenizer.additional_special_tokens_ids[
processor.tokenizer.additional_special_tokens.index("")
]

def __call__(self, examples):
texts = []
images = []
for example in examples:
image = example["image"]
question = example["query"]['en']
answer = random.choice(example["answers"])
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "Answer briefly."},
{"type": "image"},
{"type": "text", "text": question}
]
},
{
"role": "assistant",
"content": [
{"type": "text", "text": answer}
]
}
]
text = processor.apply_chat_template(messages, add_generation_prompt=False)
texts.append(text.strip())
images.append([image])

batch = processor(text=texts, images=images, return_tensors="pt", padding=True)

labels = batch["input_ids"].clone()
labels[labels == processor.tokenizer.pad_token_id] = self.image_token_id
batch["labels"] = labels

return batch

data_collator = MyDataCollator(processor)

Step 5: Setup training parameters

Configure the hyperparameters to train the model.

from transformers import TrainingArguments, Trainer

training_args = TrainingArguments(
output_dir = "IDEFICS_DocVQA",
learning_rate = 2e-4,
fp16 = True,
per_device_train_batch_size = 2,
per_device_eval_batch_size = 2,
gradient_accumulation_steps = 8,
dataloader_pin_memory = False,
save_total_limit = 3,
evaluation_strategy ="steps",
save_strategy = "steps",
eval_steps = 10,
save_steps = 25,
max_steps = 25,
logging_steps = 5,
remove_unused_columns = False,
push_to_hub=False,
label_names = ["labels"],
load_best_model_at_end = False,
report_to = "none",
optim = "paged_adamw_8bit",
)
trainer = Trainer(
model = model,
args = training_args,
data_collator = data_collator,
train_dataset = train_dataset,
eval_dataset = eval_dataset
)

Step 6: Start Training

trainer.train()

It took me about an hour to fine-tune the model for 25 steps using a T4 GPU on Google Colab. The more training steps and examples you use, the better the results tend to be. Given the limitations of the T4, that’s the best I could train the model on.

Training Results

Step 7: Evaluate the model

Now, it’s time to evaluate the model on a test example.

test_example = eval_dataset[0]
test_example["image"]
test_0.jpg
model.eval()

image = test_example["image"]
query = test_example["query"]


messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "Answer briefly."},
{"type": "image"},
{"type": "text", "text": query}
]
}
]


text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=[text.strip()], images=[image], return_tensors="pt", padding=True)
generated_ids = model.generate(**inputs, max_new_tokens=64)
generated_texts = processor.batch_decode(generated_ids[:, inputs["input_ids"].size(1):], skip_special_tokens=True)
print(generated_texts)Question: What the location address of NSDA?
Answer: [‘1128 SIXTEENTH ST., N. W., WASHINGTON, D. C. 20036’, ‘1128 sixteenth st., N. W., washington, D. C. 20036’]

Question: What the location address of NSDA?
Answer: [‘1128 SIXTEENTH ST., N. W., WASHINGTON, D. C. 20036’, ‘1128 sixteenth st., N. W., washington, D. C. 20036’]

Address of NSDA

That’s fantastic — the model has provided an accurate answer to the question posed from the test image.

Step 8: Push the fine-tuned model on Hugging Face 🤗

It’s a good idea to push the model on HF if you want to access it in the future without needing it to retrain.

# Login to your HF account.
from huggingface_hub import notebook_login
notebook_login()

from huggingface_hub import whoami
from pathlib import Path

# Output directory.
output_dir = "IDEFICS_DocVQA"
repo_name = "IDEFICS2-DocVQA-fine-tuned"
username = whoami(token=Path("https://medium.com/root/.cache/huggingface/"))["name"]
repo_id = f"{username}/{repo_name}"
from huggingface_hub import upload_folder, create_repo

repo_id = create_repo(repo_id, exist_ok=True).repo_id

upload_folder(
repo_id=repo_id,
folder_path=output_dir,
commit_message="Pushed the IDEFICS2 fine-tuned model.",
ignore_patterns=["step_*", "epoch_*"],
)

That’s all folks. Congratulations on successfully fine-tuning the Idefics2–8B model on a custom dataset. I hope you learned valuable insights into creating an image-text dataset from scratch and fine-tuning a Vision Language Model.

You can find the complete Colab notebook and the raw dataset on this GitHub repository. If you found the blog helpful, please consider giving the repository a ⭐.

I can’t wait to see what you’ll create by fine-tuning Idefics2–8B. If you have any questions or want to share something incredible that you’ve built, feel free to connect with me on LinkedIn.

Thank you for reading the blog. Stay tuned for more fascinating projects and articles headed your way.

References & Resources


[ML Story] Fine-tune Vision Language Model on custom dataset was originally published in Google Developer Experts on Medium, where people are continuing the conversation by highlighting and responding to this story.

The post [ML Story] Fine-tune Vision Language Model on custom dataset appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/05/08/ml-story-fine-tune-vision-language-model-on-custom-dataset/feed/ 0
Twinning Titans: System Design vs. Design System — Let the Showdown Begin! https://prodsens.live/2024/05/08/twinning-titans-system-design-vs-design-system-let-the-showdown-begin/?utm_source=rss&utm_medium=rss&utm_campaign=twinning-titans-system-design-vs-design-system-let-the-showdown-begin https://prodsens.live/2024/05/08/twinning-titans-system-design-vs-design-system-let-the-showdown-begin/#respond Wed, 08 May 2024 17:19:59 +0000 https://prodsens.live/2024/05/08/twinning-titans-system-design-vs-design-system-let-the-showdown-begin/ twinning-titans:-system-design-vs.-design-system — let-the-showdown-begin!

Twinning Titans: System Design vs. Design System — Let the Showdown Begin! 🚨 “ أنا مش كريم انا كرم، أنا مش…

The post Twinning Titans: System Design vs. Design System — Let the Showdown Begin! appeared first on ProdSens.live.

]]>
twinning-titans:-system-design-vs.-design-system — let-the-showdown-begin!

Twinning Titans: System Design vs. Design System — Let the Showdown Begin! 🚨

“ أنا مش كريم انا كرم، أنا مش كرم أنا كريم 🫡 “

Are you finding yourself scratching your head when someone mentions “System Design” versus “Design System”? You’re not alone! In the fast-paced world of technology, these terms can sometimes get mixed up. Let’s clear the air and break down the distinctions between these two crucial concepts.

System Design: The Blueprint of Tech Solutions

When we talk about System Design, think of it as the architect’s blueprint for a skyscraper. It’s all about planning and structuring complex systems to solve real-world problems efficiently. In the realm of tech, this often involves designing software architectures, networks, or infrastructure setups.

Key Points:

⏩ System Design focuses on the overall structure and functionality of a technological solution.

⏩ It involves making decisions about technology choices, scalability, security, and performance optimization.

⏩ Architects and engineers collaborate to create robust systems that meet specific requirements and objectives.

Design System: Crafting Consistent User Experiences

Now, let’s shift gears to Design System. Picture it as a well-organized toolbox filled with reusable design elements and guidelines. Design Systems are all about maintaining consistency and harmony across user interfaces and experiences. They ensure that every button, color, and font aligns seamlessly with your brand’s identity.

Key Points:

⏩ Design System is a collection of reusable components, patterns, and standards for building user interfaces.

⏩ It promotes consistency, efficiency, and a unified brand identity across different applications or platforms.

⏩ Designers and developers collaborate to create and maintain a cohesive set of design assets and guidelines.

📌 How to Keep Them Straight

Remember the “Big Picture” versus “Pixel Perfect” analogy:

– System Design deals with the big picture: how all the pieces fit together to create a robust and efficient technological solution.

– Design System focuses on the pixel-perfect details: ensuring that every element of the user interface is consistent, intuitive, and aligned with the brand.

So, the next time you hear someone mention System Design or Design System, you’ll know exactly what they’re talking about! Whether you’re building the next tech marvel or fine-tuning user interfaces, understanding these distinctions is key to success in the ever-evolving world of technology.

Next post will talk about Design System in #Android especially in #JetpackCompose , Stay tuned 🚨!


Twinning Titans: System Design vs. Design System — Let the Showdown Begin! 🚨 was originally published in Google Developer Experts on Medium, where people are continuing the conversation by highlighting and responding to this story.

The post Twinning Titans: System Design vs. Design System — Let the Showdown Begin! appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/05/08/twinning-titans-system-design-vs-design-system-let-the-showdown-begin/feed/ 0
An Introduction to Google Tag Manager https://prodsens.live/2024/04/02/an-introduction-to-google-tag-manager/?utm_source=rss&utm_medium=rss&utm_campaign=an-introduction-to-google-tag-manager https://prodsens.live/2024/04/02/an-introduction-to-google-tag-manager/#respond Tue, 02 Apr 2024 22:20:36 +0000 https://prodsens.live/2024/04/02/an-introduction-to-google-tag-manager/ an-introduction-to-google-tag-manager

Using Google Tag Manager effectively first requires a comprehensive understanding of how it works and what it can…

The post An Introduction to Google Tag Manager appeared first on ProdSens.live.

]]>
an-introduction-to-google-tag-manager

Using Google Tag Manager effectively first requires a comprehensive understanding of how it works and what it can be used for. This introductory guide to GTM will clear up any confusion or fears you had about using the tool, arming you with all the knowledge you need to get started.

The post An Introduction to Google Tag Manager appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/04/02/an-introduction-to-google-tag-manager/feed/ 0
Construction Bids: Mastering Construction Bidding https://prodsens.live/2024/02/27/construction-bids-mastering-construction-bidding/?utm_source=rss&utm_medium=rss&utm_campaign=construction-bids-mastering-construction-bidding https://prodsens.live/2024/02/27/construction-bids-mastering-construction-bidding/#respond Tue, 27 Feb 2024 19:20:34 +0000 https://prodsens.live/2024/02/27/construction-bids-mastering-construction-bidding/ construction-bids:-mastering-construction-bidding

Construction projects are won through the construction bidding process. In a nutshell, construction bidding involves a project owner…

The post Construction Bids: Mastering Construction Bidding appeared first on ProdSens.live.

]]>
construction-bids:-mastering-construction-bidding

Construction projects are won through the construction bidding process. In a nutshell, construction bidding involves a project owner who wants to build a construction project and contractors who are able to provide construction services.

Managing the construction bidding process and expanding your customer base without breaking the bottom line is a delicate art. Whether a seasoned construction professional or new to the business, there’s always more to learn about submitting a construction bid!

What Is Construction Bidding?

Construction bidding describes the process of bid submission and approval between a construction company and its customer, but it can also describe how subcontractors get work from the contractor once a job has been taken. Construction bidding can also be referred to as the construction tender process.

A construction bid lives or dies on accuracy. Using blueprints, construction plans and material quantity takeoffs, the bidder must estimate a realistic cost (including a profit margin) to make the job viable.

What Is a Construction Bid?

A construction bid is part of the process of submitting a proposal for a construction project. The construction bid shows potential customers that your organization is the right contractor for the job—meaning you’ll take on building and/or managing their building or structure.

In order to have an accurate forecast and competitive bid, you need tools to help estimate costs, time and other resources. ProjectManager is project management software that offers planning, scheduling and tracking tools to ensure you keep to your bid during construction management. Use our unlimited file storage to manage your construction bidding documents. Once your bid wins, you can start planning your construction projects with our Gantt charts, project calendars, kanban boards and more. Get started for free.

ProjectManager's Gantt chart
After you win your bid, track your project with powerful scheduling software. Learn more!

Construction bids are often confused with similar construction documents like construction proposals, construction quotes and construction estimates. Here are the main differences between them so you don’t confuse them.

Construction Bids vs. Construction Proposals

While construction bids and construction proposals are very similar, there’s one key difference between them. Construction bids are used when the conditions of a construction project are well-defined such as its scope of work, drawings, specifications and documentation.

In these cases, the owner is simply looking for a general contractor that has the capabilities to execute the work within the timeline and budget that’s been outlined in a construction plan by architects, engineers and the construction project management team.

A construction proposal, on the other hand, is used when some project details are unknown or left open so that contractors can create a proposal that they think best serves the needs of the project owner. The two documents that project owners might use to gather construction bids and proposals are an invitation for bid (IFB) or a request for proposal (RFP).

Construction Bids vs. Construction Quotes

A construction quote is a document that shows the estimated costs for labor or materials for completing a construction project, a portion of it or a specific task. One unique characteristic of construction quotes is that they’re only valid for a limited amount of time due to the ever-changing price of commodity resources such as lumber and fluctuating labor costs.

Construction quotes can be used in a variety of ways. For example, a construction project owner might request a quote from a construction contractor, or a contractor may ask for a quote to find out the cost of materials from its supplier.

Construction Bids vs. Construction Estimates

While a bid is an estimate, they are not exactly the same thing. There’s no hard and fast rule, but most of the time when a contractor is talking about an estimate, they’re referring to the costs of materials and labor for their project. The construction bid is what’s sent to the customer as the final, fixed price for the entire job.

Regardless of the difference, estimates need to be accurate in order not to lose money if the bid is accepted.

Get your free

Construction Proposal Template

Use this free Construction Proposal Template for Word to manage your projects better.

 

What Should Be Included In a Construction Bid?

There’s no one-size-fits-all style to a construction bid, however, certain elements are required when submitting one. When making a construction bid, be sure to include the following:

  • Contact Information: This might seem obvious, but a bid will be discarded if there’s no contact information for the construction company that submits it or the potential customer to whom it’s addressed. The basic information is name, address, phone and/or email and a place for signatures, either to acknowledge receipt or agreement on the terms. Include the location of the construction project as well.
  • Scope: Next comes an overview of the project before getting into pricing details. Here, outline the scope of the project, the services provided, the schedule for the work, necessary materials and other features required to get the job done. This section should be detailed and include customer expectations, any subcontractors that will be hired, removal of trash, visitor rules, safety protocols and more.
  • Existing Conditions: This is where to describe the job site as it currently is after conducting a preliminary site assessment. List the conditions of the site as it stands now, what action will be taken to respond to these conditions and who will be responsible for what. Also, note how conditions discovered after the start of the project will be dealt with and who will decide on the scope and cost of any remediation.
  • Cost: Estimate the price for the entire project here. Break down the total cost into subsections, including labor and materials. Add a section on costs if the project goes over the scope to manage the customer’s expectations.
  • Terms of Payment: Detail how you’ll get paid for the job if selected by the customer. Most will not pay the total cost upfront, so decide what down payment you require and the frequency of installments for the balance. These payments can be tied to various milestones in the project, which should be identified.
  • Relevant Documentation: Because of the size of most construction projects, it’s critical to the bid that any sub-projects are identified, detailed and an owner who is responsible for them determined. This will help the customer know who is responsible for what when the project begins. It also defines who has legal authority when it comes to signing documents.
  • Schedule: The work schedule will be detailed here, from the start to the end and all milestones in between. It’s a good idea to determine the extent to which you’re liable for any events that cause delays in the execution of the project. Those events could be inclement weather, delayed permits, etc. Block out the calendar, noting working days, vacation, supply lead time, zoning approval and other third-party processes that can impact the timeline.

Construction Bid Template

This construction proposal template allows you to gather all the elements you’ll need to prepare a construction bid or proposal, such as the scope of work, costs, work schedule and timeline for your project. It’s easy to use and can help you streamline the construction tender process whether you’re a construction project owner, construction firm, contractor or subcontractor.

construction bid template

 

Construction Bidding Process

When involved in the construction bidding process, you should ideally have accurate estimates and a low bid. Many customers will only look at the bottom line and go with the lowest bidder. However, don’t bid yourself out of business.

Keeping that in mind, it’s best to follow these steps in the construction bidding process.

  1. Solicitation: Referred to as a request for proposal (RFP), this is when an owner puts out a request for bids for a project they want to execute. This means asking companies to bid with all of the materials listed in the previous section.
  2. Subcontract: The general contractor will solicit bids from subcontractors for parts of the project, though this can often wait until the contractor has won the bid.
  3. Submission: The proposal will have a deadline. Once the contractor has detailed the information required by the customer, they submit the bid before or on the deadline.
  4. Selection: There’s a period when the customer goes through all the submitted bids. They will then choose the one that best fits their needs, which will be the winner of the bid.
  5. Contract: Because of the details of the bid, it can be used by both parties as a legally binding contract once agreed upon. However, it’s more likely the customer and the winner of the bid will finalize the terms and conditions sketched out in the bid and create a legal contractual agreement they both sign.
  6. Project Begins: After all of this, the project will begin and follow the agreed-upon schedule and pricing of the bid.

Types of Construction Bidding

There are different types of construction bidding, also known as construction tenders which allow project owners to specify what type of contractors and subcontractors may send bids and other guidelines for their bidding process. There are four main types of tender in construction, open, negotiated, selective and serial tendering. Here’s a quick overview of them.

Open Tendering

This type of construction tender allows any contractor to apply for a construction project. The main benefit of using an open tender process for your project is that it will allow you to gather many construction bids, which helps you find the best contractor for your projects, whether you’re looking for a low-cost bid, an experienced contractor or the best price-quality ratio.

Open tendering is the preferred method for construction government projects because it promotes fair competition, ensures transparency and helps decision-makers select the bid that best fit the needs of their community.

In the US, city and state governments are constantly accepting bids for a variety of public construction projects, which are often posted on their websites or authorized construction bidding websites that show available open tenders and all the information you’ll need to send your bids.

Negotiated Tendering

This is a special type of tender that’s used whenever the project owner has decided he’d like to proceed with a particular contractor and proceeds to only receive a bid from him and then continue to negotiate its price, terms and specifications.

Selective Tendering

A selective construction tender allows a small group of contractors to apply for a construction bidding process, which combines the simplicity of negotiated tendering, but widens its scope so you can get different types of bids and have a better understanding of what your project might cost.

Serial Tendering

Serial tendering consists in opening a construction bidding process for multiple related projects that are expected to be completed in the future, such as when a construction firm wants to build a series of apartment complexes with similar characteristics.

The main benefit from this type of tender is that it allows both the construction owner and contractor to take advantage of economies of scale for resources such as materials and equipment and it simplifies the construction bidding process.

How to Bid On Construction Jobs

Before we dive into the steps of the construction bidding process, it’s important to know the key elements of construction bidding. Whether you’re a project owner or a contractor, you’ll need to know about these elements.

There are several types of project delivery and procurement methods, as well as different types of construction contracts from which you can choose. Each has pros and cons depending on the specific requirements of your project, so you’ll need to evaluate which of them works best for you. The following are the most popular methods and contracts for your construction bidding process.

1. Define Your Project Delivery Method

A project delivery method defines the responsibilities and risks assumed by both the project owner, construction managers and contractors when starting a construction project.

  • Design Bid Build: This project delivery method consists of a design phase where a designer creates construction documents. Then those documents are released to contractors who will estimate the project costs and present a bid to the project owner. The project owner then will evaluate bids and select a contractor.
  • Design-Build: An alternative method to design-bid-build in which the contractor assumes responsibility for the design and construction of the project.
  • Construction Manager at Risk: In the construction manager at risk method, the construction manager commits to delivering the construction project at a guaranteed maximum price (GMP).

2. Select a Construction Procurement Method

Here’s a brief explanation of the main responsibilities of the project owner and contractors under the two most popular construction procurement methods.

  • General Contracting: The project owner hires a general contractor who will build the construction project. The contractor executes the construction work and might hire subcontractors to cover certain areas.
  • Construction Management: The project owner hires multiple subcontractors for each type of construction work such as plumbing, electrical work or masonry. This requires a huge effort from the owner and is suggested for bigger construction firms.

3. Choose a Contract Model

The project owner and contractor must agree on a contract model to establish a legally binding agreement for the execution of the construction project. Here are some examples.

  • Cost Plus Fee Contract: A cost plus fee contract requires contractors to report costs as they occur rather than deducting them from a budget. Then the project owner pays for the accumulated costs plus a fixed fee that’s agreed with the contractor before executing the project.
  • Guaranteed Maximum Price (GMP) Contract: In a GMP contract, the contractor defines a guaranteed maximum price for the project owner. The contractor must commit to this price even at his own expense if costs exceed his construction estimates.
  • Time and Material Contract: In a time and materials contract (t&m), the project owner pays an hourly rate for labor and purchases materials as needed by the contractor. It’s a good alternative when the project scope is uncertain and when close supervision is needed.

construction management ebook ad

Construction Bidding Best Practices

The construction bidding process is highly competitive and there can be dozens of businesses vying for one job. How do you differentiate yourself and win the work without losing money in the process?

Use Construction Bidding Websites

Various websites show construction tenders for a particular area. For example, constructconnect shows a wide variety of construction tenders located in the state of Texas.

Some organizations post their own construction tenders such as the Texas Park and Wildlife, but some sites act as intermediaries between private construction owners, contractors and subcontractors.

Know the Competition

It helps to know who else is bidding on the job. Know the competitors and what they’re doing. It’s helpful to network or join building trade groups to stay updated on what others in the construction industry are doing. When there’s a proposal, it doesn’t hurt to be the first one to bid, so keep an eye on marketplaces where bids are posted.

Be Judicious With Your Bidding

Although it helps to be the first to bid, it’s not ideal to bid on every job with a proposal. Spend time making the best proposal for the job while also ensuring it fits what your organization can accomplish at a profit. Cultivate a niche and look for work that will result in repeat business.

Build Relationships

Business is better executed on the foundation of a strong relationship, so strive to build relationships with people who are in a position to make decisions on awarding work to your company. Identify the individuals responsible for pulling the trigger on new work and develop trust with them.

Accentuate Your Strengths

When in the construction bidding process, don’t be bashful. Promote the qualities your business has that make it the right fit for the customer. Show off the team and be sure to let them know you have the experience necessary to do the job right. Sometimes value is more important than the price tag.

Take Your Time

Take your time when making the bid. Make sure to do the research and explain how you can give them the best return on their investment. If you know anyone who has worked for the customer in the past, talk to them to get an idea of what the customer wants.

More Construction Project Management Templates

Construction projects involve creating a variety of documents which help the project management team have a better understanding and control over all the different areas of the project.

Making all these documents from scratch can not only be challenging but also time consuming, which is why we’ve created dozens of free project management templates for Excel and Word you can use for construction. Here are some of them.

Construction Estimate Template

This free construction estimate template for Excel allows you to list down all your project costs such as materials, labor, equipment rental or any other cost categories you’d like to include.

Construction Schedule Template

This construction schedule template allows you to use an interactive Gantt chart to map out your project tasks on a timeline, identify dependencies, schedule resources, find the critical path of your project and much more.

ProjectManager & Construction Bids

ProjectManager is award-winning software used to organize plans that are productive and easy to share. Building your bid is akin to starting a project; you need to define a budget, manage resources, create a schedule and show it to your potential customer in the hopes that they accept your bid.

Manage All Your Construction Bids In One Place

ProjectManager’s kanban board is the perfect tool to track all the construction bids your organization is either evaluating or applying for. A kanban board has columns that allow you to group your construction bids together by their status, such as sent, in review and approved. ProjectManager’s kanban boards allow you to communicate with your team members in real time and attach files with unlimited storage.

ProjectManager's kanban board showing construction bids

Lay Out Your Project on Gantt Charts

When you’ve done the due diligence and are ready to create a schedule, our online Gantt chart is there to organize all your tasks. Add a start and end date to each task and they populate a visual timeline that shows you the entire project on one page.

Create Dependencies and Milestones

At this point, you can link all four types of task dependencies to avoid bottlenecks that can delay the execution phase of the project. Plus, set milestones as diamond icons on the timeline to mark when one phase ends and another begins. Milestones can also be used to mark any important dates you don’t want to slip through the cracks during the heat of the project.

dependencies and milestones on a Gantt chart

Unlimited File Storage for Documentation

Construction projects are complicated and involve lots of documentation. Our online tool has unlimited file storage that acts as a central hub for your project documents, making it easy to find when you need it. No more messy desks or cluttered email folders to search through. You can share the project with your customer and, if they accept your bid, they’ll be notified by email and in-app alerts on the project’s progress.

ProjectManager is an online tool that gives you live data for better decisions. Keep your customers updated with a real-time dashboard that automatically calculates project variance, costs and more and displays them in shareable graphs and charts. Plan, monitor and report on your construction project, while getting instant status updates from the job site. Try our software today with this free 30-day trial.

The post Construction Bids: Mastering Construction Bidding appeared first on ProjectManager.

The post Construction Bids: Mastering Construction Bidding appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/02/27/construction-bids-mastering-construction-bidding/feed/ 0
SUNDAY REWIND: A well-rounded personal growth plan https://prodsens.live/2024/01/21/sunday-rewind-a-well-rounded-personal-growth-plan/?utm_source=rss&utm_medium=rss&utm_campaign=sunday-rewind-a-well-rounded-personal-growth-plan https://prodsens.live/2024/01/21/sunday-rewind-a-well-rounded-personal-growth-plan/#respond Sun, 21 Jan 2024 11:25:28 +0000 https://prodsens.live/2024/01/21/sunday-rewind-a-well-rounded-personal-growth-plan/ We look back at advice from Mind the Product’s managing director Emily Tate on how product people might…

The post SUNDAY REWIND: A well-rounded personal growth plan appeared first on ProdSens.live.

]]>
We look back at advice from Mind the Product’s managing director Emily Tate on how product people might develop a personal growth plan in this week’s Sunday Rewind. Read more »

The post SUNDAY REWIND: A well-rounded personal growth plan appeared first on Mind the Product.

The post SUNDAY REWIND: A well-rounded personal growth plan appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/01/21/sunday-rewind-a-well-rounded-personal-growth-plan/feed/ 0
Profile Card Hover Effect https://prodsens.live/2024/01/16/profile-card-hover-effect/?utm_source=rss&utm_medium=rss&utm_campaign=profile-card-hover-effect https://prodsens.live/2024/01/16/profile-card-hover-effect/#respond Tue, 16 Jan 2024 21:25:07 +0000 https://prodsens.live/2024/01/16/profile-card-hover-effect/ profile-card-hover-effect

Check out this Pen I made!

The post Profile Card Hover Effect appeared first on ProdSens.live.

]]>
profile-card-hover-effect

Check out this Pen I made!

The post Profile Card Hover Effect appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/01/16/profile-card-hover-effect/feed/ 0
Photon Lockdown | HackTheBox Write-up https://prodsens.live/2024/01/11/photon-lockdown-hackthebox-write-up/?utm_source=rss&utm_medium=rss&utm_campaign=photon-lockdown-hackthebox-write-up https://prodsens.live/2024/01/11/photon-lockdown-hackthebox-write-up/#respond Thu, 11 Jan 2024 13:24:52 +0000 https://prodsens.live/2024/01/11/photon-lockdown-hackthebox-write-up/ photon-lockdown-|-hackthebox-write-up

Introduction This challenge is fairly simple. What we mainly needed to do was figure out the filetype of…

The post Photon Lockdown | HackTheBox Write-up appeared first on ProdSens.live.

]]>
photon-lockdown-|-hackthebox-write-up

Introduction

This challenge is fairly simple. What we mainly needed to do was figure out the filetype of the rootfs file using the file command. After that, we just needed to do some basic research on how to extract the data from a SquashFS file (unsquashfs). Using grep on the entire extracted filesystem, we were able to locate the file that the flag was in.

First Impressions

After downloading the challenge file and unzipping the contents, we see three files listed as so: fwu_ver, hw_ver, and rootfs.

Upon a little research, we can see that the file named rootfs is an abbreviation for root file system.

Using ls paired with the -l flag, we can see more information about the three files. What I’m mainly interested in is the size of each file:

Image description

We can see that the the file rootfs is by far the biggest, and this is probably due to the fact that it is a root file system.

We are first going to inspect the rootfs file. Since there are no visible extensions for the file, we are going to use the file command to gain more information about the type of file it is, as well as some basic information regarding the file’s contents.

Image description

From this, we can see that it is a Squashfs filesystem that has been compressed.

SquashFS

After some basic research, it was found that SquashFS is a compressed read-only file system.

In order to restore the filesystem to a more readable format, we need to extract the filesystem from rootfs.

After some more research, it was found that we can do this using the unsquashfs command. For this to work however, we need to run the command as a super user. This can be accomplished with the sudo command plus the command we wish to execute.

Image description

After we successfully run the command, we are given a new directory called squashfs-root containing the extracted filesystem from rootfs.

After changing directories to the extracted filesystem using cd

Image description

GREP

I figured to find the flag, I would just use the grep command to find the regular expression HTB, since that is what HackTheBox flags start with. By pairing the grep command with -l (list files that contain a match) and -r (recursive mode to traverse directories), I searched the entire filesystem for an expression that contained HTB, which would be the starting characters for the challenge’s flag.

Image description

This yielded three files with a match: bin/ip, bin/tc, and etc/config_default.xml. Using cat on the first two yielded nothing but machine code that is unreadable. The third one (etc/config_default.xml), however, was in plaintext.

Image description

I then decided to use grep again for HTB by piping the output of cat on the config_default.xml.

Image description

This yielded the flag we were looking for, as well as valuable login information.

The post Photon Lockdown | HackTheBox Write-up appeared first on ProdSens.live.

]]>
https://prodsens.live/2024/01/11/photon-lockdown-hackthebox-write-up/feed/ 0
Understanding React useContext https://prodsens.live/2023/11/26/understanding-react-usecontext/?utm_source=rss&utm_medium=rss&utm_campaign=understanding-react-usecontext https://prodsens.live/2023/11/26/understanding-react-usecontext/#respond Sun, 26 Nov 2023 02:25:21 +0000 https://prodsens.live/2023/11/26/understanding-react-usecontext/ understanding-react-usecontext

useContext is one of the React Hooks. As discussed in the previous blog post on React’s Pros and…

The post Understanding React useContext appeared first on ProdSens.live.

]]>
understanding-react-usecontext

useContext is one of the React Hooks. As discussed in the previous blog post on React’s Pros and Cons, the one-way data flow in React ensures a clean data flow architecture. Data from a parent referred to as props, can only be passed down to its child components and not vice versa.

However, this approach can become cumbersome when dealing with the need to pass props down through multiple layers of components, creating a “prop-drilling” scenario. This involves passing data from parent to child, then from that child to another child, and so on—a sort of bucket relay.
This is where useContext proves to be incredibly useful.

Image description

Image description

Images are from react.dev

Create the context in a separate file:
Explanation:
Contexts are defined in separate files to adhere to the principles of React and a component-based approach. This allows for a cleaner and more organized structure in your application.

Purpose:
The creation of contexts, and the subsequent use of useContext, is not merely to avoid manually passing props. Instead, it serves the purpose of establishing a context-based communication channel within the component tree. This is particularly valuable for managing global states or sharing common functionality seamlessly across components.

import { createContext } from 'react';

const SomeContext = createContext(defaultValue)

Provide it by wrapping the components:
Explanation:
To make the context value accessible to a part of the component tree, you use the Provider component. The Provider is responsible for passing down the context value through its value prop.

Purpose:
The Provider wraps the component or components where you want the context to be available, ensuring that they have access to the shared context value.

** If you call the context from another page
import { MyContext } from './MyContext.js';


  { Your components }

Use it in child components:
Explanation:
Similar to useState(), the useContext() hook is employed to access the context value within functional components. When you call useContext(MyContext), it returns the current context value for the specified context object. This hook is particularly useful when working with contexts created using React.createContext(). The context can store not only data but also functions, providing a flexible mechanism for sharing functionality across components.

Purpose:
By using useContext(), child components can seamlessly access the context value without the need for prop drilling. This enhances code readability and simplifies the passing of values between components.

import { useContext } from 'react';

const MyComponent = () => {
  const myData = useContext(MyContext);
};

The reference says,

The returned value is always up-to-date. React automatically re-renders components that read some context if it changes.

Whenever the value of a context changes, React automatically re-renders all the children components that are using that particular context. This re-rendering process ensures that the components stay in sync with the updated context value.

While this approach maintains overall data accuracy, it can become a drawback as the app grows larger.

In such cases, useReducer seems a potential solution to address this challenge. In the next article, I’ll delve deeper into this topic, and focus on utilizing both useReducer and useContext.

The post Understanding React useContext appeared first on ProdSens.live.

]]>
https://prodsens.live/2023/11/26/understanding-react-usecontext/feed/ 0
A Beginner’s Guide to Google Search Console https://prodsens.live/2023/11/20/a-beginners-guide-to-google-search-console/?utm_source=rss&utm_medium=rss&utm_campaign=a-beginners-guide-to-google-search-console https://prodsens.live/2023/11/20/a-beginners-guide-to-google-search-console/#respond Mon, 20 Nov 2023 12:24:41 +0000 https://prodsens.live/2023/11/20/a-beginners-guide-to-google-search-console/ a-beginner’s-guide-to-google-search-console

Discover the value of Google Search Console for SEO, marketing, and business owners. Learn how this free tool…

The post A Beginner’s Guide to Google Search Console appeared first on ProdSens.live.

]]>
a-beginner’s-guide-to-google-search-console

Discover the value of Google Search Console for SEO, marketing, and business owners. Learn how this free tool provides insights into your site’s performance, engagement, and technical issues. Learn how you can use GSC to make more informed decisions for your website.

The post A Beginner’s Guide to Google Search Console appeared first on ProdSens.live.

]]>
https://prodsens.live/2023/11/20/a-beginners-guide-to-google-search-console/feed/ 0