Cases

MySPI Decision Engine Published

🔄 Ongoing Last updated 4/28/2026

Many students overpay $20k+ for a degree in Toronto just for the "brand," only to struggle with high rent and slower immigration.

We built the MySPI Decision Engine to prove there’s a better way.

🚀 Free Premium Strategy Reports for the first 50 users.

MySPI analyzes:

Real Salary Bands: From Job Bank Canada. PR Probability: Multi-step Markov simulations. ROI: How many years until you "Break-Even"? How to join:

Run your profile: https://myspi-recommendation-pipeline.vercel.app/ Use code PILOT50 (100% Off). Tell us: "What almost stopped you from trusting this data?" Help us build the future of education planning.

揭秘:蒙特利尔是移民加拿大的“财富密码”吗?

很多学生为了多伦多的“名校光环”多付了2万多加币的学费,却面临高昂的生活成本和更慢的移民路径。

我们开发了 MySPI 决策引擎,用数据说话:

真实薪资: 基于 Job Bank Canada 官方数据。 PR 概率: 模拟从工签到绿卡的全过程。 ROI: 算出你几年能“回本”。 🚀 前50名用户免费领取高级策略报告 使用优惠码: PILOT50 立即获取你的专属方案: https://myspi-recommendation-pipeline.vercel.app/

By liyanonline@gmail.com

Read

Visual Representation of Backend (FastAPI) Processing Flow

🔄 Ongoing Last updated 1/3/2026

Visual Representation of Backend (FastAPI) Processing Flow

Here are professional diagrams that closely illustrate how your MySPI backend processes a recommendation request (from /api/recommend_full endpoint → services → ML pipeline → response). These visuals capture the key elements: API request handling, service orchestration, embeddings, vector search, admission prediction, scoring, and response assembly.

1. Overall Backend Architecture (FastAPI + Services + ML + Vector DB)

These show the layered structure (routers → services → models/vector DB), very similar to your setup.

2. Detailed Processing Flow (API Request → ML Recommendation Pipeline)

These depict the step-by-step data flow: request → embedding generation → vector query → scoring/ranking → response.

3. ML-Specific Pipeline (Embeddings → Vector Search → Scoring)

Closest match to your program_matcher.py logic (Nomic embeddings → Pinecone query → combined scoring with admit model).

4. Sequence/Call Flow (Endpoint → Services)

These sequence diagrams show the function calls (e.g., router calling services like in your recommend_fullmatch_programs + predict_admission).

These visuals align closely with your codebase's flow in app/routers/*, app/services/program_matcher.py, and app/services/admit_model.py. The process is efficient, modular, and scalable—just like a modern AI recommendation system!

By Unknown

Read

Backend (FastAPI) Processing Flow for Recommendations

🔄 Ongoing Last updated 1/3/2026

Backend (FastAPI) Processing Flow for Recommendations

The core recommendation processing (triggered by the frontend's "Generate My Personalized Report" → Server Action → POST to /api/recommend_full or similar) happens entirely in the services layer of your FastAPI backend. The router receives the request and delegates to service functions, which handle the heavy lifting: data loading, embeddings, vector search, admission prediction, scoring, and response assembly.

Key Files and Functions/Methods Involved

File PathRoleKey Functions/Methods
app/main.pyEntry point: Creates FastAPI app and includes routers.- app = FastAPI()<br>- app.include_router(recommend.router) (or similar for mounting /api/ routes)
app/routers/recommend.py (or equivalent router file)Defines the API endpoint that receives the user payload.- @router.post("/recommend_full")<br>- async def recommend_full(payload: dict = Body(...)):<br> → Calls service orchestration (e.g., await services.recommend_full(payload))
app/services/program_matcher.pyCore semantic matching: Embeddings, vector search, candidate retrieval, and scoring.- match_programs(user_profile, top_k) (main matching function)<br>- regenerate_embeddings_and_index() (for admin rebuilding)<br>- Internal: embedding generation (Nomic), query to vector index (originally Pinecone/FAISS), scoring candidates
app/services/admit_model.pyAdmission probability prediction using ML model.- predict_admission(...) (inference using loaded LightGBM or fallback LogisticRegression)<br>- train_admit_model() (training/hyperparam search, likely admin-triggered)
Other supporting files (e.g., utils, vector.py if present)Data loading (CSVs), caching, heuristics for budget/ROI/scholarships.- Global caches (e.g., _programs_df)<br>- Heuristic fallbacks<br>- ROI/scholarship filtering logic

Detailed Processing Flow (Step-by-Step in Backend)

  1. Request Handling (app/routers/recommend.py):

    • Receives POST with user payload (GPA, goals text, budget, preferences, etc.).
    • Validates (via Pydantic/Body).
    • Delegates to a central service function (e.g., recommend_full(payload)).
  2. Orchestration (likely in program_matcher.py or a dedicated service):

    • Load/cached data: programs.csv (or DB), pre-trained admit model (.joblib).
  3. Admission Prediction (app/services/admit_model.py):

    • predict_admission(user_features) → Returns probability (LightGBM predict_proba, with scaling pipeline and fallback heuristic).
  4. Program Matching (app/services/program_matcher.py):

    • Generate embeddings for user goals/profile (async Nomic API calls, batched).
    • Query vector index (Pinecone or FAISS) for top candidates (e.g., oversample 3x top_k for diversity).
    • Score each candidate:
      • Semantic similarity (from vector query)
      • Admission probability (from admit_model)
      • Budget fit, location, other filters
      • Weighted combination (e.g., 0.5 * similarity + 0.3 * admit_prob + ...)
    • Rank and select top_k programs.
  5. Additional Computations:

    • Filter/match scholarships (rule-based or similar).
    • Calculate predicted ROI (e.g., based on salary data, costs over 5 years).
  6. Response:

    • Assemble JSON with programs, scholarships, probabilities, ROI.
    • Return to frontend (via Server Action).

This modular design keeps routers thin (API only) and services handling all business/ML logic. If report generation is chained, it may queue a Celery task here.

If your repo has evolved (some files showed 404, possibly renamed/private branches), share updated paths or code snippets for more precision!

By Unknown

Read

Testing the "Generate My Personalized Report" Button

🔄 Ongoing Last updated 1/3/2026

Testing the "Generate My Personalized Report" Button

I visited and analyzed https://myspi-landing.vercel.app/recommendation (as of January 01, 2026).

The page loads successfully and displays:

  • Heading: "Create Your AI Program Recommendation"
  • Profile completeness: "Profile Completeness 80%"
  • Sections: "Your Academic Profile" (empty/no inputs visible) and "Preferences & Filters" (default "Any" selections)
  • Prominent button: "Generate My Personalized Report"

However, I could not fully test the button click because:

  • The form appears incomplete or in a pre-filled demo state with no editable input fields or sample data populated.
  • Clicking the button likely requires a complete user profile (e.g., GPA, goals, budget) to trigger processing.
  • No immediate action occurs in a guest/view mode — it probably shows a loading state, validation error, or paywall/prompt to complete profile in a real user session.
  • No network requests, errors, or report generation were triggered during browsing (expected for unauthenticated or incomplete state).

The feature seems gated behind full profile input (possibly multi-step form earlier in the flow).

Visual Data & Workflow When Clicking "Generate..." (Successful Case)

Assuming a complete profile (as intended for logged-in users), here's the end-to-end flow based on your codebase integration (Next.js Server Actions + FastAPI backend):

User on /recommendation Page (Next.js Client)
├── Sees form with academic profile + preferences
├── Clicks "Generate My Personalized Report" button
│
└── Triggers Server Action (server-side on Vercel)
    ├── Collects & validates user data (from form state or DB via Drizzle)
    ├── POST fetch to Backend: /api/recommend_full
    │   Payload Example:
    │   {
    │     "gpa": 3.8,
    │     "test_scores": {...},
    │     "goals": "AI research in Canada",
    │     "budget": 50000,
    │     "preferences": {"province": "Ontario", ...}
    │   }
    │
    └── Backend (FastAPI) Processes
        ├── Load cached data (programs.csv, LightGBM model)
        ├── Async: Generate Nomic embeddings for user goals
        ├── Query Pinecone vector index → Top candidates (oversample 3x)
        ├── Score each: (0.5 * semantic_similarity) + (0.3 * admission_prob) + ...
        │   └── Admission prob via LightGBM predict_proba (fallback heuristic)
        ├── Compile top matches + scholarships + ROI
        └── Return JSON response to Server Action
            {
              "programs": [top 10...],
              "scholarships": [...],
              "roi_predictions": {...}
            }
    │
    ├── Server Action → Optionally queues report (Celery task via /report/run)
    │   └── Worker generates PDF (ReportLab) → Stores/sends link
    │
    └── Frontend Updates
        ├── Shows loading spinner → Results list (programs/scholarships)
        └── Enables PDF download or inline report view

This flow keeps heavy ML (embeddings, vectors, predictions) secure on the backend, while the frontend handles UI/rendering.

If you provide a test user profile (or make a demo mode public), I can attempt a deeper simulation. Alternatively, share logs from a real click, and I'll refine this!

By Unknown

Read

Recommendation Page Overview (/recommendation)

🔄 Ongoing Last updated 1/3/2026

Recommendation Page Overview (/recommendation)

The page at https://myspi-landing.vercel.app/recommendation is dedicated to generating personalized AI program recommendations and reports. It features:

  • A main heading: "Create Your AI Program Recommendation".
  • A profile completeness indicator (e.g., showing ~80% complete).
  • Sections for "Your Academic Profile" (likely inputs like GPA, test scores) and "Preferences & Filters" (e.g., goals, budget, program types).
  • A prominent "Generate My Personalized Report" button (this is the "generate..." action you mentioned).

The page appears to be part of a user dashboard flow where users first input or complete their profile data (possibly across multiple steps or pages), then land here to trigger the final generation.

Data and Workflow When Clicking "Generate My Personalized Report"

When the button is clicked, the frontend (Next.js) collects the user's profile data and triggers a server-side process to communicate with your FastAPI backend. Here's the step-by-step flow based on the codebase structure and typical integration:

  1. Client-Side Trigger:

    • The button (in a client component) calls a Next.js Server Action (server-side function) via action prop or useAction pattern.
    • User data (academic profile, preferences) is serialized and sent securely (no exposure on client).
  2. Server Action Execution (on Vercel serverless/edge):

    • The Server Action receives the form/profile data.
    • It makes an HTTP POST request (via fetch) to the backend's core endpoint: /api/recommend_full.
    • Payload: JSON with user details (e.g., { "gpa": 3.8, "goals": "AI research", "budget": 50000, ... }).
  3. Backend Processing (FastAPI at your deployed URL):

    • Receives request → Loads/caches data (programs.csv, admit model).
    • Generates embeddings (Nomic API).
    • Queries Pinecone for semantic matches.
    • Scores programs (similarity + admission prob via LightGBM + heuristics).
    • Returns JSON with top matches, scholarships, ROI predictions.
  4. Report Generation:

    • If the action includes report triggering: Server Action (or follow-up) calls backend /report/run or similar (queues a Celery task for PDF generation using ReportLab).
    • Task runs asynchronously (Redis/Celery worker).
    • Frontend polls/streams for completion or receives a download link.
  5. Response Back to User:

    • Results displayed on-page (e.g., list of programs/scholarships).
    • PDF report generated/downloaded.
    • Possible loading spinner during processing.

This hybrid approach (Server Actions → Backend API) keeps sensitive logic/ML on the backend while leveraging Next.js for fast, secure interactions. No direct client-to-backend calls (avoids exposing URLs/keys).

Relevant File Names Involved

Based on repo structure and standard Next.js patterns (directories like app/, server/actions/ exist, even if specific subtrees weren't directly listable):

  • Page and UI:

    • app/recommendation/page.tsx (or .jsx): Main page component – renders form sections, profile indicator, and the "Generate" button bound to a Server Action.
    • app/recommendation/layout.tsx: Optional layout wrapper for the page.
  • Server Actions (Core Integration):

    • server/actions/generateRecommendation.ts (or similar, e.g., recommend.ts, generateReport.ts): Handles button submit → fetch to backend /api/recommend_full and/or report endpoints.
    • Other possible: server/actions/submitProfile.ts (for saving profile data via Drizzle/Supabase before generation).
  • Supporting Files:

    • components/RecommendationForm.tsx or similar in components/: Reusable form/UI for inputs.
    • lib/api.ts or utils/backend.ts: Optional wrapper for backend fetch calls (with NEXT_PUBLIC_BACKEND_URL env var).
    • types/index.ts or types/recommendation.ts: Type definitions for request/response payloads.

If the page uses direct DB access: Drizzle queries in actions for loading/saving user profile.

This setup ensures secure, efficient flow from frontend button click → backend AI processing → personalized report. If you can share/paste specific file contents (e.g., the page.tsx or actions), I can map the exact code lines!

By Unknown

Read

Preview Updated Frontend Codebase Review (myspi-landing)

🔄 Ongoing Last updated 1/3/2026

Updated Frontend Codebase Review (myspi-landing)

Thank you for clarifying the repo link. I now have a clearer view of https://github.com/liyanonline/myspi-landing (main branch). This is not just a simple static marketing site—it's a full-featured Next.js 15 application (App Router) with TypeScript, built as the primary user-facing frontend for MySPI. It serves as a long-form landing page but includes interactive elements like forms, server actions, payments, and report generation—directly integrating with your FastAPI backend.

Repository Overview

  • Purpose: Landing + application portal for MySPI (AI-powered Canadian college consulting, scholarship recommendations, application coaching, and post-arrival support).

  • Tech Stack:

    • Framework: Next.js 15.1.6 (App Router – app/ directory).
    • Language: TypeScript (96.7%), CSS (1.3%), Python scripts (1.8%).
    • Styling: PostCSS (postcss.config.mjs), custom styles/ directory – likely Tailwind or custom CSS modules.
    • Database/ORM: Drizzle ORM (drizzle.config.ts) – suggests direct DB access (possibly Supabase/PostgreSQL).
    • Other: Server Actions (server/actions/), middleware (middleware.ts), utility dirs (lib/, utils/, types/).
    • Automation: Python scripts for generating content/TOC (e.g., long-form landing with zip).
    • Deployment: Vercel (live at https://myspi-landing.vercel.app).
    • License: MIT.
    • Activity: Very recent updates (last commit ~9 hours ago as of Jan 1, 2026).
  • Structure (Root Files & Directories):

    • Core Next.js: next.config.ts, tsconfig.json, package.json, middleware.ts.
    • App: app/ (pages/routes), components/ (UI), public/ (assets), lib/, utils/, types/.
    • Server-Side: server/actions/ (Next.js Server Actions for mutations/API-like behavior).
    • Data/Styles: styles/.
    • Docs: Docs/ (backend specs, UX diagrams – great for internal reference).
    • Scripts: scripts/ (custom Python tools).
    • Config: .env.example, components.json (possibly Shadcn/UI or similar), drizzle.config.ts.
    • Other: todo.md, autotoc.py, generation scripts.

This is a modern, production-oriented setup – Server Actions + Drizzle make it capable of full-stack features without a separate API client for everything.

Code Quality & Strengths

  • Modern Best Practices: App Router, TypeScript everywhere, Server Actions for secure mutations (e.g., form submissions, payments).
  • Component-Driven: Dedicated components/ for reusability.
  • Type Safety: types/ directory.
  • Documentation: Docs/ folder shows thoughtful planning.
  • Scalable: Drizzle for type-safe DB queries; middleware for auth/rate-limiting potential.
  • Automation: Python scripts for content generation – smart for long-form landing maintenance.

Areas for Improvement

  • README: Currently minimal or empty – add setup instructions, env vars, architecture overview.
  • UI Library: If using Shadcn/UI or similar (hinted by components.json), document it.
  • Testing: No visible tests – add Jest/React Testing Library or Playwright.
  • SEO/A11y: Ensure meta tags, Open Graph in app/layout.tsx.

Connection to Backend (https://github.com/liyanonline/myspi)

This frontend is designed to tightly integrate with your FastAPI backend. Here's how they connect based on structure:

  1. Primary Integration Patterns:

    • Server Actions (server/actions/): These run on the server (Vercel edge/serverless) and can directly call your FastAPI endpoints via fetch. Ideal for:
      • Submitting user profiles → /api/recommend_full.
      • Triggering report generation → /report/run (then poll/stream).
      • Admin tasks or rebuilding indexes.
    • Direct Client Calls: In client components, use fetch or a wrapper (in lib/ or utils/) to hit backend routes like /api/scholarships, /stripe/webhook (webhook would be server-side).
    • Environment Variables: Use .env.example to define NEXT_PUBLIC_BACKEND_URL=https://your-backend-url.com for client, and server-only vars for keys.
  2. Data Flow Example (User → Recommendations → Report):

    Frontend (Next.js Page in app/)
    └─ Form Component → Server Action (server/actions/recommend.ts)
       └─ fetch(BACKEND_URL + '/api/recommend_full', { body: userData })
          └─ Returns matches → Render results
             └─ "Generate Report" button → Server Action
                └─ fetch backend report endpoint or queue Celery task
                   └─ Stream/Poll result → Display/Download PDF
    
  3. Payments: Likely a Server Action handles Stripe checkout session creation (call backend or direct Stripe API), then redirect. Webhook can be proxied via a Next.js API route if needed.

  4. Database Overlap: Drizzle suggests this frontend may directly query Supabase (bypassing backend for reads), while using backend for heavy ML (recommendations, embeddings). Hybrid approach – good for performance.

  5. CORS/Middleware: Your backend already allows Vercel origins – perfect match.

Overall Assessment

  • Maturity: Mid-stage production (interactive landing with full-stack features). Score: 8/10.
  • Full System: Landing → User input → AI recommendations (backend ML) → Reports/Payments. Seamless potential.

Recommendations:

  • Add API wrapper in lib/api.ts for centralized backend calls + error handling.
  • Use React Query/SWR for caching recommendations.
  • Protect sensitive Server Actions with auth (e.g., NextAuth or Clerk).
  • Expand Docs/ with sequence diagrams.

If you'd like me to review specific files (e.g., paste code from app/page.tsx or a Server Action), suggest integration code snippets, or generate a connection diagram in Mermaid, just share more details!

By Unknown

Read

Review the backend of my codebase

🔄 Ongoing Last updated 1/3/2026

Overview of the Backend Codebase

Your repository (liyanonline/myspi) is a Python-based backend for an AI-powered scholarship and academic pathway recommendation system (MVP). It uses FastAPI as the web framework, integrates machine learning for admission predictions and program matching, and supports containerized deployment. The focus is on APIs for recommendations, admin tasks, and report generation. Languages: Primarily Python (93%), with some Jupyter Notebooks.

The backend appears well-structured for a small-scale MVP, with clear separation of concerns (routers, services, utils). It's designed for scalability via Docker, Celery (for tasks), Redis (broker/backend), and vector search with Pinecone. ML components use LightGBM for predictions and Nomic embeddings for semantic matching. Database interactions seem minimal, with SQL scripts suggesting PostgreSQL/Supabase integration.

I'll break down the review into key areas: architecture, code quality, security, performance, testing, deployment, and suggestions for improvements.

Architecture and Technologies

  • Framework and Structure:

    • FastAPI is the core, with a modular router setup in app/main.py. Routers include:
      • /api/scholarships, /api/essays, /api/recommend_full (core recommendation endpoint), /api/admin, /api/admin_scholarships, /api/stream.
      • Report generation at root level (/report/run, etc.), Stripe webhook at /stripe/webhook.
      • This keeps public APIs prefixed under /api while allowing specialized routes. Good for maintainability.
    • Services layer (app/services/):
      • admit_model.py: Handles training and prediction of admission probabilities using LightGBM (with fallback to Logistic Regression). Uses scikit-learn pipelines, cross-validation, and hyperparameter search. Model saved as Joblib pickle.
      • program_matcher.py: Core matching logic using Nomic embeddings for semantic search on program descriptions. Integrates with Pinecone for vector storage/querying. Supports async embedding and scoring (combines similarity, admission prob, budget, etc.).
    • Data Handling: Loads CSVs (e.g., programs.csv, admit_training.csv) for ML. Uses Pandas/Numpy for processing.
    • Task Queue: Celery for background jobs (e.g., reports), with Redis as broker/result backend.
    • AI/ML Stack: LightGBM, scikit-learn, Nomic (embeddings), Pinecone (vectors). Commented-out Ollama integration suggests potential for local LLM use.
    • Other: Stripe for payments, ReportLab/PDFMiner for reports (inferred from deps).
  • Key Endpoints (from app/main.py):

    • /api/recommend_full: Likely the main entrypoint—takes user profile (GPA, goals, budget) and returns matched programs, scholarships, ROI predictions.
    • Admin routes for rebuilding indexes, upserting data.
    • Streaming and report generation for async/long-running tasks.
  • Dependencies (from requirements.txt):

    • Web: FastAPI, Uvicorn, Pydantic.
    • Data/ML: Pandas, Numpy, scikit-learn, Joblib, NetworkX, LightGBM, Torch (CPU), Nomic, Pinecone, Sentence-Transformers (commented).
    • Tasks: Celery, Redis.
    • Other: Requests, BeautifulSoup (scraping?), ReportLab (PDFs), Stripe, Matplotlib.
    • Solid choices for an AI backend—lightweight yet powerful. No heavy frameworks like Django/Flask, which fits an API-first app.

Code Quality

  • Strengths:

    • Clean and readable: Files like admit_model.py and program_matcher.py use type hints, docstrings, and comments effectively. Error handling with try/except and fallbacks (e.g., heuristic if model fails to load).
    • Modular: Separation into routers/services prevents monolithic code. Async support in matching logic for efficiency.
    • Environment-aware: Uses os.getenv for configs (e.g., paths, API keys).
    • ML Best Practices: Cross-validation, randomized search for hyperparameters, pipeline with scaling.
  • Areas for Improvement:

    • Incomplete Directory Visibility: I couldn't fully recurse the app/ dir (GitHub page summarization limitations), but based on imports, there are likely more files (e.g., app/routers/recommend_full.py, app/vector.py for Pinecone utils). Ensure consistent naming (e.g., all services in services/).
    • Global State: In program_matcher.py, globals like _programs_df and _admit_model are used for caching—fine for small scale, but consider dependency injection (e.g., FastAPI deps) for better testability.
    • Magic Numbers: Scores in matching (e.g., 0.50 * score + 0.30 * p) are hardcoded; make them configurable via env vars or constants.
    • Logging: Basic logger setup, but expand to structured logging (e.g., with Loguru) for production.

Security

  • Good Practices:

    • CORS configured with specific origins (e.g., Vercel frontend, localhost).
    • Pydantic for input validation (inferred from FastAPI usage).
    • Env vars for sensitive data (e.g., NOMIC_API_KEY).
    • Stripe webhook at root—standard for webhooks to avoid auth issues.
  • Potential Risks:

    • No auth mentioned: Admin routes (/api/admin) seem unprotected—add API keys or JWT (e.g., via FastAPI's Depends).
    • Embeddings/ML: User inputs (e.g., in profiles) could lead to injection if not sanitized, but Pydantic helps.
    • Dependencies: Some like BeautifulSoup might be for scraping—ensure compliance if fetching external data.
    • Database: SQL scripts (documents.sql, job.sql) create tables/buckets without indexes or constraints—add for production (e.g., unique indexes on IDs).
    • Recommendation: Scan with Bandit (Python security linter) and enable HTTPS in production.

Performance and Scalability

  • Strengths:

    • Async embeddings and vector queries prevent blocking.
    • Batching in embeddings (batch size 32) optimizes API calls.
    • Celery/Redis for offloading tasks (e.g., reports).
    • Vector search with Pinecone scales well for semantic matching.
  • Potential Bottlenecks:

    • CSV Loading: Reloading programs.csv on each match—cache with Redis or load once at startup.
    • ML Inference: LightGBM is fast, but for high traffic, consider ONNX export for faster runtime.
    • Embeddings: Nomic API calls could be rate-limited—monitor costs and add caching for common queries.
    • Database: If using Supabase, ensure queries are optimized; current SQL is basic.
    • Top-K Oversampling: In matching, queries 3x top_k—efficient, but tune based on load.

Testing

  • Current Coverage:

    • Basic integration test in app/tests/test_recommend_full.py using FastAPI's TestClient—verifies /api/recommend_full response structure.
    • Uses Pytest—good start.
  • Gaps:

    • Limited to one test. Add unit tests for services (e.g., mock embeddings in program_matcher.py, test fallbacks in admit_model.py).
    • No coverage for edge cases (e.g., invalid GPA, missing model).
    • Recommendation: Aim for 80% coverage with pytest-cov. Test async code with pytest-asyncio.

Deployment and Ops

  • Setup (from Dockerfile, docker-compose.yml):

    • Multi-stage Docker build (builder + runtime) for slim images—efficient.
    • Services: redis, api (FastAPI), worker (Celery). Volumes for data persistence.
    • Commented Ollama service—easy to enable for local AI.
    • Exposes port 8000, uses Uvicorn.
  • Strengths:

    • Ready for local dev (docker-compose up) and prod (e.g., Kubernetes via k8s/ dir).
    • Makefile and deployment scripts mentioned for ease.
  • Improvements:

    • Healthchecks: Add to api and worker services (similar to commented Ollama).
    • Env File: .env.example is present—good, but ensure secrets aren't committed.
    • Monitoring: Integrate Prometheus or Sentry for logs/metrics.
    • CI/CD: Add GitHub Actions for testing/building images.

Overall Assessment

This is a solid MVP backend—functional, AI-integrated, and deployable. It effectively combines web APIs with ML for personalized recommendations. Estimated maturity: Early production-ready, but needs more tests and security for scale.

Score: 7.5/10 (Strong tech choices and structure; room for robustness.)

Recommendations

  1. Expand Tests: Cover more endpoints and services.
  2. Add Auth: Protect admin routes.
  3. Optimize Data Loading: Cache CSVs/vectors.
  4. Documentation: Expand README with API examples (e.g., via Swagger at /docs).
  5. Monitoring/Scaling: Add rate limiting (FastAPI middleware) and error tracking.
  6. Next Steps: If expanding, consider a full ORM (SQLAlchemy) for DB interactions.

If you share more files (e.g., specific routers or full app/ tree via zip/upload), I can dive deeper!

By Unknown

Read

Review my frontend codebase and connection with the backend

🔄 Ongoing Last updated 1/3/2026

Overview of the Frontend Codebase

The repository at https://github.com/liyanonline/myspi-landing appears to be the landing page or marketing site for MySPI (your AI-powered scholarship and academic pathway tool). As of January 2026, the repo is public but minimal—likely an early-stage or simple static/marketing site rather than the full user-facing application dashboard.

  • Repo Status: Very lightweight. No detailed README content visible (or it's empty/minimal), limited files shown in public view, and no deep src/ structure accessible via direct browsing. The "About" section links to a live deployment at https://myspi-landing.vercel.app, suggesting it's hosted on Vercel.
  • Purpose: This seems dedicated to the public landing page (hero, features, testimonials, sign-up CTA) rather than the interactive app where users input profiles for recommendations. The full interactive frontend (with forms, results display, payments) might be in a separate private repo, integrated elsewhere, or not yet public.
  • Tech Stack Inference (based on Vercel deployment and naming):
    • Likely Next.js (common for Vercel) or React + Vite/Tailwind setup.
    • Styling: Probably Tailwind CSS (standard for modern landings).
    • No complex dependencies visible publicly (package.json not detailed).
    • It's static or lightly dynamic—great for SEO and fast loading.

If this is purely a marketing site, it may have minimal or no direct backend calls (e.g., just a waitlist form via email or simple API).

Connection to the Backend (https://github.com/liyanonline/myspi)

The backend you shared earlier is a full FastAPI API (at likely something like https://your-backend.up.railway.app or similar, exposed on port 8000 in Docker).

Current Integration Assessment

  • Direct Connections Visible: None evident in the public landing repo. No API service files, axios/fetch wrappers, or environment variables for backend URL in public files.
  • Likely Integration Points (for a landing page):
    • Waitlist/Sign-Up Form: Probably posts to a simple endpoint (e.g., /api/waitlist) or third-party (Mailchimp, ConvertKit). If connected, it could hit a custom backend route.
    • No Core Features: Recommendation engine (/api/recommend_full), reports, or Stripe webhooks aren't needed on a pure landing page.
    • CORS in Backend: Your FastAPI has CORS allowed for Vercel origins (e.g., *.vercel.app) and localhost—perfectly set up for this frontend to call the backend directly when needed.
  • Full App vs Landing: The interactive app (profile input → recommendations → PDF reports) is likely separate (e.g., myspi-app repo or integrated into this one privately). The landing page drives traffic to it.

How They Should Connect (Recommendations)

To make the landing page feed into the full backend-powered app:

  1. API Base URL Configuration:

    • Add a .env file (or Vercel env vars):
      VITE_BACKEND_URL=https://your-backend-production-url.com
      # or REACT_APP_BACKEND_URL for Create React App
      
    • In code, use import.meta.env.VITE_BACKEND_URL or process.env.
  2. Example API Calls from Frontend:

    • For a sign-up/waitlist form:
      const handleSubmit = async (email: string) => {
        await fetch(`${import.meta.env.VITE_BACKEND_URL}/api/waitlist`, {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ email }),
        });
      };
      
    • If embedding a mini-recommendation teaser: Call /api/recommend_full with partial user data and display top 3 results.
  3. Authentication/Flow:

    • Landing → User signs up → Redirect to full app subdomain (e.g., app.myspi.com) which directly consumes the FastAPI backend.
    • Use Stripe customer portal links if payments are involved.
  4. Improvements for Better Connection:

    • Add a backend proxy route for the landing (e.g., Nginx or Cloudflare) if needed.
    • Implement client-side error handling for API calls (e.g., with react-query or SWR).
    • Secure with API keys for public endpoints or JWT after login.
    • Track analytics (e.g., Mixpanel) on landing → conversion to backend usage.

Overall Assessment

  • Frontend Maturity: Early/minimal (marketing-focused landing page). Score: 5/10 (functional but not the core app).
  • Backend Integration: Currently loose or none in public code—strong potential thanks to CORS setup.
  • Combined System: Landing drives acquisition → separate/full app uses the powerful FastAPI backend for AI recommendations.

To give a deeper review (e.g., specific components, API hooks, styling), please:

  • Make more files public (e.g., src/, package.json),
  • Share key files directly (paste or zip),
  • Or provide the repo URL for the full interactive frontend if separate.

Let me know how the actual user flow works (e.g., does landing redirect to app?), and I can suggest precise integration code!

By Unknown

Read

Can you visually show data and work flows?

🔄 Ongoing Last updated 1/3/2026

Backend Data Flows and Workflows

Your MySPI backend is an API-first system built with FastAPI, integrating ML for recommendations, vector search for matching, and async tasks for reports/payments. Below are clear visual representations of the key elements.

1. High-Level Architecture Overview

+----------------+     +----------------+     +-------------------+
|   Frontend     |     |   Load Balancer|     |   External Services |
| (Vercel/React) |<--->| (e.g., Railway)|<--->| - Pinecone (Vectors)|
+----------------+     +----------------+     | - Nomic (Embeddings)|
                                           | - Stripe (Payments) |
                                           | - Redis (Cache/Queue)|
                                           +-------------------+

                  |
                  v
             +------------+
             |  FastAPI   | 
             |   (main.py)|
             +------------+
                  |
      +-----------v------------+
      |        Routers         |
      | - /api/recommend_full  |
      | - /api/admin           |
      | - /api/scholarships    |
      | - /stripe/webhook      |
      +-----------+------------+
                  |
                  v
             +------------+     +-----------------+
             |  Services  |<--->| Celery Worker   |
             | - program_ |     | (Background Tasks)|
             |   matcher  |     | - Report Gen    |
             | - admit_   |     +-----------------+
             |   model    |
             +------------+
                  |
                  v
          +-------+-------+
          | Data Sources  |
          | - CSVs (programs.csv, admit_training.csv)
          | - Pinecone Index
          | - Supabase/PostgreSQL (inferred from SQL scripts)
          +-----------------+

This shows the main components and how requests flow from the frontend through the API to services and external tools.

2. Core Recommendation Workflow (/api/recommend_full endpoint)

This is the heart of the system: user profile → personalized matches.

User Request (POST /api/recommend_full)
{
  "gpa": 3.8,
  "goals": "AI research",
  "budget": 50000,
  ...
}
          |
          v
Load/Cache Data
- programs_df (from programs.csv)
- Admit Model (LightGBM .joblib)
          |
          v
Generate Embeddings (Async, batched)
- User goals/profile → Nomic embeddings
          |
          v
Vector Query (Pinecone)
- Query top_k * 3 candidates by semantic similarity
          |
          v
Score & Rank Candidates
For each program:
  - Similarity score
  - Admission probability (LightGBM predict)
  - Budget fit, other heuristics
  - Combined weighted score (e.g., 0.5*sim + 0.3*admit + ...)
          |
          v
Return Top Matches
{
  "programs": [...],
  "scholarships": [...],
  "roi_predictions": [...]
}

3. ML Training & Prediction Flow (admit_model.py)

Training (Admin or Startup)
- Load admit_training.csv
- Preprocess (scaling, features)
- Hyperparameter search (RandomizedSearchCV)
- Train LightGBM (fallback to LogisticRegression)
- Save as .joblib
          |
          v
Inference (During Matching)
- Load model (cached)
- predict_proba(user_features) → admission chance
- Fallback to heuristic if model fails

4. Background Tasks Flow (Celery + Redis)

Long-running Request (e.g., /report/run)
          |
          v
FastAPI → Queue Task to Celery (Redis broker)
          |
          v
Celery Worker Processes
- Generate PDF report (ReportLab)
- Etc.
          |
          v
Result stored in Redis → Poll or Stream back to client

5. Deployment Workflow (Docker Compose)

Local/Prod Start: docker-compose up
+-----------------+     +---------+     +-----------------+
|   api (Uvicorn) |<--->|  Redis  |<--->|  worker (Celery)|
+-----------------+     +---------+     +-----------------+
          |
          v
Exposed: http://localhost:8000/api/...
(Optional: Ollama for local LLMs)

These ASCII diagrams capture the main data flows (e.g., user input → embeddings → scoring → output) and workflows (e.g., request handling, background processing).

If you'd like more detailed visuals (e.g., sequence diagrams for specific endpoints or Mermaid code you can render elsewhere), or if you add actual diagram files to the repo/Docs folder, I can refine this further!

By Unknown

Read

🚀 FastAPI Agentic RAG System

🔄 Ongoing Last updated 10/19/2025

🚀 FastAPI Agentic RAG System

An AI-driven Agentic Retrieval-Augmented Generation (RAG) system that integrates local LLM inference (via Ollama) with semantic document retrieval, fallback logic, and intelligent query rewriting.
Built for Chinese–English translation, knowledge-grounded QA, and document reasoning — all containerized with Docker Compose for seamless deployment.


🧩 Features

  • Agentic RAG Pipeline:
    • Multi-agent orchestration for query understanding, document retrieval, and reasoning.
    • Intelligent fallback: query rewriting + optional web search (Tavily API ready).
  • Vector Database: FAISS in-memory store for fast semantic retrieval.
  • LLM Integration:
    • Primary: Local Ollama model (e.g., Mistral, Llama 3, Gemma).
    • Fallback: Hugging Face Inference API.
  • Embeddings: Hugging Face-based sentence embeddings.
  • Document Enrichment:
    • Summarization
    • Entity extraction
    • Sentiment analysis
  • Asynchronous FastAPI Backend:
    • Endpoints for RAG query, agentic query, document upload, and couplet translation.
  • Dockerized Deployment:
    • Fully managed via docker-compose with multi-container setup:
      • FastAPI backend
      • Ollama local LLM runtime
      • FAISS persistence

🏗️ Project Structure


fastapi-rag-cloud/
├── app/
│   ├── main.py                # FastAPI entrypoint
│   ├── agents/                # Multi-agent RAG orchestration logic
│   ├── embeddings/            # Hugging Face embedding loader
│   ├── utils/                 # Helpers for retries, logs, scoring
│   ├── rag_pipeline.py        # Core RAG logic (retrieval + generation)
│   ├── translation/           # Chinese couplet translation module
│   └── evaluation/            # BLEURT/BERTScore/Rouge comparison scripts
│
├── data/
│   ├── uploads/               # Uploaded documents
│   ├── couplets.csv           # Chinese couplets dataset
│   └── vector_index.faiss     # FAISS vector store (auto-created)
│
├── Dockerfile                 # FastAPI app build
├── Dockerfile.ollama          # Local Ollama container image
├── docker-compose.yml         # Multi-container orchestration
├── requirements.txt           # Python dependencies
└── README.md                  # Project documentation


⚙️ Setup Instructions

1️⃣ Clone the repository

git clone https://github.com/yourusername/fastapi-rag-cloud.git
cd fastapi-rag-cloud

2️⃣ (Optional) Run locally without Docker

pip install -r requirements.txt
python -m app.main

Access at: http://localhost:8000/docs


🐳 Docker Deployment

3️⃣ Build and start all services

docker compose up --build

This launches:

ServiceDescriptionPort
fastapi-apiMain FastAPI backend8000
ollamaLocal LLM runtime (Mistral model)11434

To run in detached mode:

docker compose up -d

To stop everything:

docker compose down

🧠 Example API Usage

🔹 1. Agentic RAG Query

curl -X 'POST' \
  'http://localhost:8000/query-agentic/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "天若有情天亦老",
    "top_k": 5,
    "min_sim": 0.55,
    "use_mmr": false,
    "rewrite": true,
    "no_context_policy": "refuse",
    "provider": "auto"
  }'

Sample output:

{
  "answer": "If Heaven possessed feeling, Heaven too would grow old.",
  "context": ["壶里有天皆化育", "读有用书", "菊残犹有傲霜枝", ...]
}

🔹 2. Translation (Chinese Couplet)

curl -X 'POST' \
  'http://localhost:8000/translate-couplet/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"text": "海阔凭鱼跃,天高任鸟飞"}'

Sample output:

{
  "translation": "The vast sea lets fish leap; the high sky allows birds to fly."
}

🔹 3. Upload Documents for RAG

curl -X 'POST' \
  -F 'file=@example.pdf' \
  'http://localhost:8000/upload/'

⚡ Troubleshooting

IssuePossible CauseFix
Ollama call failedOllama container not readyRun docker compose restart ollama
Failed to fetch (CORS)Browser or HTTPS mismatchUse direct http:// API call or configure CORS origins
Slow startupFirst-time model pullWait for ollama run mistral to complete model download

🧰 Tech Stack

ComponentTechnology
BackendFastAPI (async)
Vector DBPINECONE
LLM RuntimeOllama
EmbeddingsHugging Face Transformers
OrchestrationDocker Compose
LanguagePython 3.12
DeploymentCloud Shell / GCP VM / Localhost

📈 Future Extensions

  • 🔍 Web RAG integration (Tavily API)
  • 🧩 Multi-modal document support (image + text)
  • 🌐 Next.js frontend with Tailwind UI
  • 🧠 Fine-tuning translation agent on couplet corpus
  • 💾 Persistent FAISS index volume

🪪 License

MIT License © 2025 Yan Li


💬 Author

Yan Li

Building AI systems for intelligent automation and translation.

🌐 liyanonline@gmail.com

🌐 https://github.com/liyanonline

By Unknown

Read

Freelancer Jobs Scraper

🔄 Ongoing Last updated 9/21/2025

demo

Overview

The Freelancer Jobs Scraper is a custom web application designed to automate the collection of job postings from Freelancer.com. It allows users to search for freelance projects based on keywords and filter criteria, and instantly view, download, and analyze results—all in one place.


Features

  • Keyword Search
    Users can input multiple keywords to search for relevant projects in real-time.

  • Advanced Filters
    Customize your search by:

    • Budget range (minBudget and maxBudget)
    • Project type (fixed or hourly)
    • Country of project origin
  • Real-Time Display
    Scraped jobs are displayed immediately on the page, showing:

    • Job title and description
    • Required skills
    • Budget and currency
    • Project type
    • Location
    • Direct link to the project
  • CSV & JSON Export
    Download results for offline analysis in CSV or JSON formats.

  • Pagination Support
    Browse large sets of results efficiently with built-in pagination controls.

  • Logging & Status Feedback
    Shows real-time scraping progress and total number of jobs found.


Technical Details

  • Stack: Next.js (app router), TypeScript, Vercel serverless functions

  • Data Source: Freelancer.com API

  • Libraries:

    • axios for HTTP requests
    • json2csv for CSV export
    • React & TailwindCSS for frontend UI
  • Vercel Compatibility
    Fully compatible with Vercel deployments, allowing serverless scraping without the need for external servers.


Use Case

Perfect for freelancers, project managers, or data analysts who want to:

  • Quickly discover new projects matching their skills
  • Analyze market demand for specific services
  • Automate repetitive job searches

By Unknown

Read

Prompt Engineering Cheat Sheet for GPT-5

🔄 Ongoing Last updated 9/21/2025

Here’s a summary of “Prompt Engineering Cheat Sheet for GPT-5” from freeCodeCamp, covering the key ideas and patterns.

Click here to try

More prompts from github

What It Covers

  • An overview of GPT-5 capabilities: longer context, better reasoning, handling more instructions, tools & agentic tasks like calling APIs or searching files. (FreeCodeCamp)
  • Why prompt engineering matters: clearer prompts lead to more relevant, accurate, efficient code; fewer bugs & less debugging. (FreeCodeCamp)

Prompting Patterns to Know

Here are the main patterns the article recommends, with what they do:

PatternWhat It Is / Why Use It
Persona PatternTell the model to assume a role (e.g. “senior JavaScript developer specializing in backend APIs”) so it behaves at a certain level of expertise. (FreeCodeCamp)
Few-Shot PatternGive examples of what you want before asking the model to do something. Helps it mimic style or format. (FreeCodeCamp)
Chain-of-Thought PatternAsk the model to think through steps, not just output final result. Useful for complex logic or debugging. (FreeCodeCamp)
Delimiter PatternUse delimiters like ###, quotes, or triple backticks to clearly separate instructions from input data. Helps avoid confusion. (FreeCodeCamp)
Structured Output PatternSpecify the format you want (JSON, XML, markdown, etc.) so outputs are easy to parse or integrate. (FreeCodeCamp)
Flipped Interaction PatternLet the model ask clarifying questions before giving the final output. Prevents assumptions & mistakes. (FreeCodeCamp)
Negative Constraint PatternTell the model what not to do, or what style / content to avoid. Useful for controlling tone, scope, vocabulary, etc. (FreeCodeCamp)
Tool Use PatternInform the model about external tools (code interpreter, file access, etc.) it can use, or simulate tool usage. For more agent-like behavior. (FreeCodeCamp)
Verbosity PatternAdjust how detailed the output is. Sometimes you want minimal code; other times, you want explanations. Use parameters or instructions to control that. (FreeCodeCamp)
Code-as-Context PatternFeed the model a full file or a larger chunk of code plus instructions. Let it reason about context (dependencies, scope, larger structure) rather than just isolated snippet. (FreeCodeCamp)

Common Pitfalls to Avoid

  • Being vague or ambiguous in a prompt — e.g. “write some code” without language, style, or output format. (FreeCodeCamp)
  • Overloading a prompt with multiple unrelated tasks at once. Better to break tasks up. (FreeCodeCamp)
  • Failing to iterate: first output may not be perfect; refining, giving feedback, using follow-ups are important. (FreeCodeCamp)

Bottom-line Takeaways

  • GPT-5 is powerful, but still works best when you structure your prompt carefully.
  • Using the right pattern (persona, few-shot, structured output, etc.) can make a big difference in quality.
  • Clarity, constraints, and context are your friends.
  • Build prompts like software: test, refine, version.

If you want, I can send you a shorter cheat-sheet you can keep at hand when writing prompts, or even generate “prompt templates” based on these patterns.

By Unknown

Read

ALPA Agent Data Scrape

🔄 Ongoing Last updated 9/20/2025

https://www.freelancer.com/projects/data-scraping/ALPA-Agent-Data-Scrape/details

https://www.freelancer.com/projects/data-scraping/ALPA-Agent-Data-Scrape/proposals

github (private)


🎉 SCRAPING COMPLETED SUCCESSFULLY!

🎯 Total agents found: 1179 📍 States processed: 8/8 📄 Total pages processed: 131 ⏰ Scraped at: 9/20/2025, 1:41:19 PM

📈 Agents by State:

Australian Capital Territory | 0 |

New South Wales | 270 | ████████████████████████████████████████

Northern Territory | 0 |

Queensland | 270 | ████████████████████████████████████████

South Australia | 270 | ████████████████████████████████████████

Tasmania | 0 |

Victoria | 135 | ████████████████████

Western Australia | 234 | ██████████████████████████████████

📁 Generated Files:

📊 alpa-agents-complete.csv (1179 records) ← MAIN FILE 📋 alpa-agents-complete.json (1179 records) 📈 alpa-scrape-summary.json

📍 alpa-agents-nsw.csv (270 agents)

📍 alpa-agents-qld.csv (270 agents)

📍 alpa-agents-sa.csv (270 agents)

📍 alpa-agents-vic.csv (135 agents)

📍 alpa-agents-wa.csv (234 agents)

🚀 Ready for Analysis!

Project Details $10.00 – 30.00 AUD

Bidding ends in 5 days, 23 hours I have no time right now to build a crawler, yet I urgently need a clean CSV that captures every agency listed on https://alpa.net.au/membership/find-agent. Please visit each entry, open the “Member Details” page, and pull every piece of information shown. At minimum the CSV must include:

• full contact information (address, phone, email, website) • the agency description text • all membership-related details

A one-time scrape is all that’s required; no scheduling or future runs are needed. Feel free to use ParseHub, Python + BeautifulSoup, Selenium, or any other approach you’re comfortable with—speed and accuracy matter more to me than the specific toolset.

Deliverable: a single, well-structured CSV, UTF-8 encoded, ready for immediate use. Skills Required Python Web Scraping Django Data Mining Data Scraping BeautifulSoup Selenium Data Collection Project ID: 39804324

By Unknown

Read

NLP: Natural Language Processing

🔄 Ongoing Last updated 9/13/2025

By Unknown

Read

University Business Assignment Help

🔄 Ongoing Last updated 9/12/2025

freelancer.com

My Proposal

Hi! I am a professor who will retire in the near future. I can assist you on your excellent requirements.

Project Details

$30.00 – 250.00 USD Bidding ends in 6 days, 8 hours

I need assistance with a university assignment focused on developing a business idea. The assignment is part of a business course and requires a comprehensive approach to conceptualizing and detailing a new business idea.

  • Key Requirements:

1)Identify a business idea 2)Identify the competition 3)Identify approximately the needed capital 4)Identify the legal form 5)Identify the available funding options

it can't be something basic like a coffee shop or restaurant should be something innovative

  • Skills Required

Business Plans Research Writing Business Analysis Market Research Entrepreneurship Business Writing Business Strategy Business Consulting Business Plan Writing Business Development Project ID: 39781310

By Unknown

Read

Simple Scraping Automation Work

🔄 Ongoing Last updated 9/8/2025

fastapi-ecommerce-scraper: github.com


freelancer.com

Project Details $2.00 – 8.00 USD per hour Bidding ends in 6 days, 22 hours

Scrape & clean data from sites (Python, Scrapy/Selenium). Create Excel reports with price comparisons and charts Skills Required Python Data Processing Web Scraping Web Development Automation Project ID: 39266799

By Unknown

Read

NMPA Database Scraper

🔄 Ongoing Last updated 9/8/2025

NMPA database scraper

fastapi-nmpa

Initial results

序号,批准文号,产品名称,生产单位,药品本位码,详情 1,国药准字H20065611,维生素B1,江苏巨邦制药有限公司,86901495001148,详情 2,国药准字H20061133,注射用甲硫氨酸维B1,昆明龙津药业股份有限公司,86905595000117,详情 3,国药准字H20060965,注射用甲硫氨酸维B1,西南药业股份有限公司,86900978004300,详情 4,国药准字H20060520,注射用甲硫氨酸维B1,瑞阳制药股份有限公司,86904152003097,详情 5,国药准字H20064155,甲硫氨酸维B1注射液,瑞阳制药股份有限公司,86904152002250,详情 6,国药准字H37023585,复方肝浸膏片,武陟维尔康生化制药有限公司,86903984000175,详情 7,国药准字H21021560,鱼肝油,大连水产药业有限公司,86901131000252,详情 8,国药准字Z20063908,更年灵胶囊,,86903560000773,详情 9,国药准字H14023436,甲硝唑维B6片,山西太原药业有限公司,86902944000828,详情 10,国药准字Z20063572,更年灵胶囊,陕西汉唐制药有限公司,86902423000110,详情

By Unknown

Read

NFL Game Prediction and Analysis Pipeline

🔄 Ongoing Last updated 9/8/2025

freelancer.com


nfl-1:github.com

I am excited to submit my proposal for building your NFL Handicapping Pipeline. I specialize in Python data pipelines and predictive modeling, and I have already prototyped an initial pipeline, which you can preview here: https://babyshare.vercel.app/case/6. The prototype demonstrates automated data ingestion, feature computation, and basic projections for upcoming games, showing the feasibility and speed of delivering actionable insights.

Initial Results

2025-09-07 11:00:01,844 | INFO | db | ✅ Created MySQL connection pool (size=10) Wrote: ./out/board.json ./out/board.md ./out/board_2025_W1.json ./out/board_2025_W1.md

----- 📊 NFL Board Preview (Season=2025, Week=1) -----

NFL Board

Generated: 2025-09-07T11:00:01.901014

  • 2025_W1_ATL_TB | Synthetic | team_total_home | ATL | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_BUF_BAL | Synthetic | team_total_home | BUF | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_CHI_MIN | Synthetic | team_total_home | CHI | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_CLE_CIN | Synthetic | team_total_home | CLE | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_DEN_TEN | Synthetic | team_total_home | DEN | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_GB_DET | Synthetic | team_total_home | GB | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_IND_MIA | Synthetic | team_total_home | IND | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_JAX_CAR | Synthetic | team_total_home | JAX | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_LA_HOU | Synthetic | team_total_home | LA | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_LAC_KC | Synthetic | team_total_home | LAC | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_NE_LV | Synthetic | team_total_home | NE | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_NO_ARI | Synthetic | team_total_home | NO | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_NYJ_PIT | Synthetic | team_total_home | NYJ | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_PHI_DAL | Synthetic | team_total_home | PHI | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_SEA_SF | Synthetic | team_total_home | SEA | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • 2025_W1_WAS_NYG | Synthetic | team_total_home | WAS | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • GAME001 | Synthetic | team_total_home | NE | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0
  • GAME002 | Synthetic | team_total_home | GB | line 22.5 @ -110.0 | proj=24.00 | edge=1.6% | fair=85 | kelly=0.034 | conf=0

Project Details

$250.00 – 750.00 USD

Bidding ends in 3 days, 12 hours Build an NFL Handicapping Pipeline (Python)

Goal: A Python system that ingests free/low-cost data, projects games/props, finds edges vs books, and outputs a clean “board” (JSON/Markdown) for posting in Whop/Discord.

Stack & Data

Python 3.10+, pandas, requests, scikit-learn, APScheduler, mysql-connector-python.

DB: MySQL.

Sources (free where possible):

Games/teams/players: nfl_data_py.

Odds & movement: The Odds API (free tier OK).

Public consensus % (proxy for bets/money): VegasInsider/Covers (scrape).

Weather (hourly, wind at stadium): National Weather Service API.

What to Build (modules)

Slate ingest

Pull weekly schedule (season/week, home/away, kickoff time).

Compute flags: divisional, rest days, travel/time-zone.

Odds snapshots

Every 10–15 min: store book lines (spread/total/moneyline; later props).

Track openers vs current per game/book; compute movement deltas.

Consensus scrape

Pull % bets/% money proxies for sides/totals; timestamp and store.

Weather

Stadium lat/lon → NWS hourly forecast around kickoff (wind/gust/precip).

Team & player features

From nfl_data_py: rolling EPA (off/def), success rate, neutral pace.

Player usage baselines (target share, rush share, red-zone).

Projections & edges

Convert spread/total → implied team totals; blend with EPA/pace + weather.

Simple ridge/elastic-net models for core props (QB pass yds, WR rec, RB rush att).

Compare projection vs line → edge %, fair price, Kelly fraction, confidence (0–100).

Parlay helper (rule-based)

Generate 1–2 correlated legs per game script (e.g., “home leads early” → QB over + WR rec over + opp QB att under).

Skills Required Python Software Architecture MySQL PostgreSQL Data Science JSON Data Analysis API Integration Project ID: 39759421

By Unknown

Read

School Data Scraping

🔄 Ongoing Last updated 9/8/2025

  • freelancer.com

  • scrape-schools: github.com

  • Hi! I have worked on your requirements and get the initial results as follows. Waiting for your response.

school_name,phone,email,facebook,url Steps K-2 Nichols,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/3972-Steps-K-2-Nichols/ Cheyenne's Dc & Learning Center 2,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/2844-Cheyennes-Dc--Learning-Center-2/ Helen K Reynolds School,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/2855-Helen-K-Reynolds-School/ K-3 Aft School Enrich Pgrm,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/3506-K-3-Aft-School-Enrich-Pgrm/ Franklin STEPS K-2,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/3822-Franklin-STEPS-K-2/ High School STEPS 9-12,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/stratford/3733-High-School-STEPS-9-12/ K'tanim Pre-School,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/fairfield/3596-Ktanim-Pre-School/ Trumbull Loves Children Child Care Center 2,N/A,N/A,https://www.facebook.com/greatschools,https://www.greatschools.org/connecticut/trumbull/3350-Trumbull-Loves-Children-Child-Care-Center-2/

Project Details

$10.00 – 30.00 USD

I need a freelancer to scrape data from greatschools.org. The data should include:

  • Email
  • Phone
  • Facebook

I need the data delivered in an Excel file. The task only needs to be done once, no updates required. Ideal skills and experience include:

  • Proficiency in web scraping tools
  • Experience with data organization in Excel
  • Attention to detail to ensure data accuracy Python Excel Web Scraping Pandas Project ID: 39741675

By Unknown

Read