58 1 year ago

Upload an Image and Check if it is Harmful Content

vision 12b 27b
ollama run harishkumar56278/VisionContentModerator:27b

Details

1 year ago

ba691dfd0dd3 · 17GB ·

gemma3
·
27.4B
·
Q4_K_M
You are a Cybersecurity Content Moderation AI specialized in image analysis. Your task is **only** t
< Harish Kumar S , Email: harishkumar56278@gmail.com, Site: harish-nika.github.io>
{ "num_ctx": 4096, "repeat_penalty": 1.2, "stop": [ "<end_of_turn>" ], "
{{ if .System }}Moderator: {{ .System }}{{ end }} User: {{ .Prompt }} Moderator: {{ .Response }}
Gemma Terms of Use Last modified: February 21, 2024 By using, reproducing, modifying, distributing,

Readme

Cybersecurity Image Moderation Model

This repository contains a custom Cybersecurity Content Moderation AI model built using Ollama. The model is designed to analyze and classify images for harmful or inappropriate content.

Model Information

  • Base Model: gemma3:12b
  • Task: Image moderation and classification
  • Integration: Designed for use with Ollama

Features

  • Classifies images into predefined categories of harmful content.
  • Provides confidence scores (0 to 1) for each classification.
  • Generates structured JSON output for seamless integration.
  • Ensures strict system behavior to focus only on moderation.

Moderation Categories

The model classifies images into the following categories:

  1. Violence & Gore – Depictions of physical harm, blood, weapons, or self-harm.
  2. Nudity & Explicit Content – Adult content, pornography, child exploitation.
  3. Hate Symbols & Extremism – Swastikas, racial propaganda, extremist symbols.
  4. Misinformation & Deepfakes – Manipulated images designed to mislead.
  5. Illegal Activities – Drug usage, weapons, fraud-related visuals.
  6. Self-Harm Encouragement – Content glorifying or normalizing self-harm.
  7. Terrorism-Related Content – Support, promotion, or planning of terrorist acts.
  8. Spam & Scam Visuals – Fraudulent offers, deceptive content.

If no harmful content is detected, the image is classified as Safe.

Model Configuration

FROM gemma3:12b

# Set model parameters
PARAMETER temperature 0.2      
PARAMETER num_ctx 4096         
PARAMETER top_p 0.8            
PARAMETER repeat_penalty 1.2   

System Behavior

The model is configured to strictly analyze and classify images. It does not engage in discussions, explanations, or opinions.

  • If an image is submitted, it is classified into one or more categories.
  • If no category applies, it is labeled as “Safe” with a “Not Harmful Content” verdict.
  • If the input is invalid, the model returns:
    
    {
    "error": "Invalid input. Provide a valid image file."
    }
    

Example Outputs

Harmful Image

{
  "classification": {
    "violence_gore": {
      "confidence_score": 0.92,
      "justification": "Detected bloodstains and visible injuries."
    }
  },
  "max_confidence_category": "violence_gore",
  "final_verdict": "Harmful Content",
  "safe_content": false
}

Safe Image

{
  "classification": {},
  "max_confidence_category": null,
  "final_verdict": "Not Harmful Content",
  "safe_content": true
}

Usage

To load the model into Ollama, ensure you have ollama installed and run:

ollama create moderation_model -f <path_to_model_file>

To use the model for moderation:

ollama run moderation_model "<image_path>"

License

< Harish Kumar S , Email: harishkumar56278@gmail.com, Site: harish-nika.github.io >