feat: add object detector feature in modmesh #619
Conversation
modmesh/pilot/_vision.py
Outdated
| logger.setLevel(logging.DEBUG) | ||
|
|
||
| if 'model' not in globals(): | ||
| model = YOLO('./modmesh/pilot/yolo11n.pt') # Please check the model path |
There was a problem hiding this comment.
How do you get the model? Maybe have a runtime download logic?
There was a problem hiding this comment.
Yes, it will check the path if the model is exist during runtime, nor it will download it directly to the specified path.
There was a problem hiding this comment.
I just found there is a directory called thirdparty, maybe I should specify the path overthere instead?
There was a problem hiding this comment.
thirdparty is for the 3rd libraries. In this case, I think you can put the model file at the same directory of pilot runtime. Btw, it seems that the download logic is not implemented yet, right?
There was a problem hiding this comment.
ultralytics library had already done the download logic, no need to do it agin, perhaps they will find the model name is in their file server or not, then download it when trigger class YOLO initialization.
modmesh/pilot/_vision.py
Outdated
| @@ -0,0 +1,99 @@ | |||
| """ | |||
There was a problem hiding this comment.
Is it possible to write tests to validate the implementation?
There was a problem hiding this comment.
How would you recommend to place the tests, put them in the tests/test_pilot.py perhaps?
There was a problem hiding this comment.
I think you can put it at tests/test_vision.py?
yungyuc
left a comment
yungyuc
left a comment
There was a problem hiding this comment.
Please make sure CI passes before requesting for review.
- Correct copyright headers.
- Remove unnecessary code like
WrapRVisionDockWidget, which looks like a placeholder. - The pimpl class
RVisionDockWidget::Implis not necessary. Do not use pimpl. - Do not create a symbol named
modmesh::BoundingBox. - Always add an end marker to classes and namespaces.
I see you are using pybind11 to call back into Python to use YOLO. Why don't you just write PySide6 to do it?
| const uchar *data = rgbImg.bits(); | ||
| py::array_t<uint8_t> np_img({height, width, channels}, data); | ||
| py::object vision_mod = py::module_::import("modmesh.pilot._vision"); | ||
| py::object yolo_func = vision_mod.attr("yolo_detect"); |
There was a problem hiding this comment.
If you call YOLO from Python, why not use PySide?
I thought it was a must to write all the GUI component with |
|
@rockleona The code base has changed a lot. Please rebase to refresh the CI status. |
rockleona
force-pushed
the
feat/yolov11
branch
from
3a678c2 to
79ad5a8
Compare
rockleona
force-pushed
the
feat/yolov11
branch
from
79ad5a8 to
c069cd7
Compare
|
I've made a lots of changes, please find the items below:
The latest GUI will be look like this, I didn't change the layout but slightly different on the detail like model status and logging message in 
|
I cannot find a button to run the CI process, maybe it should be executed by a repo maintainer? |
You should use your own fork to test for CI.
Can you move the image preview away from the widget window (lower left) to the central sub-window, like other windows for 2D and 3D plots? |
yungyuc
left a comment
yungyuc
left a comment
There was a problem hiding this comment.
There will be some discussions before we can merge any code about YOLO.
- Include upsplash license text.
- Clean up image source link.
- Evaluate to use Qt instead of PIL.
- Discuss why including ultralytics.
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| https://unsplash.com/license |
There was a problem hiding this comment.
Make a copy of the license text here in this file.
| Test Files | ||
| ========== | ||
|
|
||
| - cat.jpg (original source: https://unsplash.com/photos/orange-and-white-cat-on-yellow-surface-sR0cTmQHPug?utm_source=unsplash&utm_medium=referral&utm_content=creditShareLink) |
|
|
||
| import numpy as np | ||
| import requests | ||
| from PIL import Image |
There was a problem hiding this comment.
If possible, I do not want to have PIL. modmesh is already using Qt which should include all image handling features that PIL provides. Please evaluate if you can simply use Qt/PySide for processing images.
| import numpy as np | ||
| import requests | ||
| from PIL import Image | ||
| from ultralytics import YOLO |
There was a problem hiding this comment.
I feel including a huge thirdparty like ultralytics defeats the principle of "doing it ourselves" in modmesh. @rockleona please elaborate why you include ultralytics.
3 participants
I started to have a research on computer vision, as the first step, this PR introduce Ultralytics YOLO as the object detection tool, and import yolov11 model as the base model.
Usage is really easy, just load an image, enable the detection, then you will see the result on the Console Widget. You can check the screenshot as below:
