Extracting Data from Photos using Image Processing Techniques
Hello everyone. In this post, we will extract data using image processing techniques. We have two images of leaves here as our example, and the goal is to process the images for machine learning classification. We will be using python libraries. Let’s start.
Overall Process
In this post, we will show how to extract data for both neural networks and traditional machine learning approaches. The overall process consists of the following steps:
- Binarization
- Extract Images using Connected Components
- Extract Region Properties DataFrame
Binarization
Binarization involves changing the image from a multitone image into a two-tone image (black and white). If we have a colored image, we must convert it first to a grayscale image. After converting the image to grayscale, we must set a threshold value to classify the pixel values as either white (1) or black (0). We can say that our image is binarized after this step.
from skimage.io import imread
from skimage.color import rgb2grayleaves = imread('plantD.jpg')
bin_leaves = rgb2gray(leaves) > 0.4
Extract Images using Connected Components
If you want to use neural networks in classifying the image, you may want to extract each leaf in the image as your training/test dataset. We can use connected components to determine the properties of each region.
As we can see from the image above, the white color is our background. We can use background = 1 when we get the connected components.
from skimage.measure import label, regionprops
def bounding_box(leaves):
#Binarization
bin_image = rgb2gray(leaves) > 0.4
label_im = label(bin_image, background = 1)
#Region Props
regions = regionprops(label_im)
fig, ax = plt.subplots(figsize=(5,5))
for props in regions:
minr, minc, maxr, maxc = props.bbox
length = maxr - minr
width = maxc - minc
area = length*width
#Drawing Boxes per Area
if area > 2000 and area < 140000:
if width > 20 and length > 20:
bx = (minc, maxc, maxc, minc, minc)
by = (minr, minr, maxr, maxr, minr)
ax.plot(bx, by, '-r', linewidth=2)
ax.imshow(leaves, cmap = "gray")
You may notice that some leaves are not detected. Applying morphological operations and other image processing techniques may help in detecting all the leaves in this image. We can use the coordinates of the bounding boxes in extracting the leaves from the image. One note on using this approach is to resize the images into uniform size before training the neural network.
Extract Region Properties DataFrame
If you want to use traditional machine learning methods (Logistic Regression, Ensemble methods, Naive Bayes) in classifying leaves, you must have an idea of how the leaves can be classified. Label region properties like area, perimeter, bounding box area, and eccentricity can be used. Derived features like area/perimeter and bounding box/area can also be used. It is up to you on how you will use the features. The code below extracts the features of the leaves including the derived variable.
from skimage.measure import label, regionprops, regionprops_tabledef region_props_dataframe(leaves):
bin_image = rgb2gray(leaves) > 0.4
label_im = label(bin_image, background = 1)
properties = ["area", "bbox_area", "convex_area","perimeter","eccentricity", "bbox", "extent"]
df = pd.DataFrame(regionprops_table(label_im, properties = properties))
df = df[df["bbox_area"] > 100]
df = df[df["bbox_area"] < 140000]
df["Area/Perimeter"] = df["area"] /df["perimeter"]
df["Area/Convex_area"] = df["area"] / df["convex_area"]
df = df.drop(["area", "bbox_area","perimeter",
"bbox-0", "bbox-1", "bbox-2",
"bbox-3", "convex_area"], axis = 1).reset_index(drop=True)
return df
In conclusion, we are successful in extracting data from photos for machine learning classification. Which of the two approaches is better? It highly depends on how will you preprocess the image beforehand. While the neural network method doesn’t require you to manually pick the feature, the results are less interpretable than traditional ML techniques.
