From 45e56350da7d84dddbf628452180d9b083df232e Mon Sep 17 00:00:00 2001 From: Vamshi Verma Date: Thu, 3 Jul 2025 20:59:25 -0400 Subject: [PATCH] Feat/performance optimization (#42) * feat: Add security and OWASP instructions This adds a new instruction file for secure coding practices based on the OWASP Top 10 to guide GitHub Copilot in generating more secure code. * fix: add YAML front matter to security instructions and correct README link casing * chore: update README.md after running update-readme.js * feat: add the most comprehensive, practical, and engineer-authored performance optimization instructions. Covers frontend, backend, and database best practices with actionable guidance, scenario-based checklists, troubleshooting, and pro tips. Updates README.md to include this new resource. * Apply suggestion from @aaronpowell * Apply suggestion from @aaronpowell --------- Co-authored-by: Aaron Powell --- README.md | 1 + .../performance-optimization.instructions.md | 420 ++++++++++++++++++ 2 files changed, 421 insertions(+) create mode 100644 instructions/performance-optimization.instructions.md diff --git a/README.md b/README.md index fecd409..80381c4 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,7 @@ Team and project-specific instructions to enhance GitHub Copilot's behavior for - [Guidance for Localization](instructions/localization.instructions.md) - Guidelines for localizing markdown documents - [Markdown](instructions/markdown.instructions.md) - Documentation and content creation standards - [Next.js + Tailwind Development Instructions](instructions/nextjs-tailwind.instructions.md) - Next.js + Tailwind development standards and instructions +- [Performance Optimization Best Practices](instructions/performance-optimization.instructions.md) - The most comprehensive, practical, and engineer-authored performance optimization instructions for all languages, frameworks, and stacks. Covers frontend, backend, and database best practices with actionable guidance, scenario-based checklists, troubleshooting, and pro tips. - [Python Coding Conventions](instructions/python.instructions.md) - Python coding conventions and guidelines - [Secure Coding and OWASP Guidelines](instructions/security-and-owasp.instructions.md) - Comprehensive secure coding instructions for all languages and frameworks, based on OWASP Top 10 and industry best practices. diff --git a/instructions/performance-optimization.instructions.md b/instructions/performance-optimization.instructions.md new file mode 100644 index 0000000..fe5707d --- /dev/null +++ b/instructions/performance-optimization.instructions.md @@ -0,0 +1,420 @@ +--- +applyTo: '*' +description: 'The most comprehensive, practical, and engineer-authored performance optimization instructions for all languages, frameworks, and stacks. Covers frontend, backend, and database best practices with actionable guidance, scenario-based checklists, troubleshooting, and pro tips.' +--- + +# Performance Optimization Best Practices + +## Introduction + +Performance isn't just a buzzword—it's the difference between a product people love and one they abandon. I've seen firsthand how a slow app can frustrate users, rack up cloud bills, and even lose customers. This guide is a living collection of the most effective, real-world performance practices I've used and reviewed, covering frontend, backend, and database layers, as well as advanced topics. Use it as a reference, a checklist, and a source of inspiration for building fast, efficient, and scalable software. + +--- + +## General Principles + +- **Measure First, Optimize Second:** Always profile and measure before optimizing. Use benchmarks, profilers, and monitoring tools to identify real bottlenecks. Guessing is the enemy of performance. + - *Pro Tip:* Use tools like Chrome DevTools, Lighthouse, New Relic, Datadog, Py-Spy, or your language's built-in profilers. +- **Optimize for the Common Case:** Focus on optimizing code paths that are most frequently executed. Don't waste time on rare edge cases unless they're critical. +- **Avoid Premature Optimization:** Write clear, maintainable code first; optimize only when necessary. Premature optimization can make code harder to read and maintain. +- **Minimize Resource Usage:** Use memory, CPU, network, and disk resources efficiently. Always ask: "Can this be done with less?" +- **Prefer Simplicity:** Simple algorithms and data structures are often faster and easier to optimize. Don't over-engineer. +- **Document Performance Assumptions:** Clearly comment on any code that is performance-critical or has non-obvious optimizations. Future maintainers (including you) will thank you. +- **Understand the Platform:** Know the performance characteristics of your language, framework, and runtime. What's fast in Python may be slow in JavaScript, and vice versa. +- **Automate Performance Testing:** Integrate performance tests and benchmarks into your CI/CD pipeline. Catch regressions early. +- **Set Performance Budgets:** Define acceptable limits for load time, memory usage, API latency, etc. Enforce them with automated checks. + +--- + +## Frontend Performance + +### Rendering and DOM +- **Minimize DOM Manipulations:** Batch updates where possible. Frequent DOM changes are expensive. + - *Anti-pattern:* Updating the DOM in a loop. Instead, build a document fragment and append it once. +- **Virtual DOM Frameworks:** Use React, Vue, or similar efficiently—avoid unnecessary re-renders. + - *React Example:* Use `React.memo`, `useMemo`, and `useCallback` to prevent unnecessary renders. +- **Keys in Lists:** Always use stable keys in lists to help virtual DOM diffing. Avoid using array indices as keys unless the list is static. +- **Avoid Inline Styles:** Inline styles can trigger layout thrashing. Prefer CSS classes. +- **CSS Animations:** Use CSS transitions/animations over JavaScript for smoother, GPU-accelerated effects. +- **Defer Non-Critical Rendering:** Use `requestIdleCallback` or similar to defer work until the browser is idle. + +### Asset Optimization +- **Image Compression:** Use tools like ImageOptim, Squoosh, or TinyPNG. Prefer modern formats (WebP, AVIF) for web delivery. +- **SVGs for Icons:** SVGs scale well and are often smaller than PNGs for simple graphics. +- **Minification and Bundling:** Use Webpack, Rollup, or esbuild to bundle and minify JS/CSS. Enable tree-shaking to remove dead code. +- **Cache Headers:** Set long-lived cache headers for static assets. Use cache busting for updates. +- **Lazy Loading:** Use `loading="lazy"` for images, and dynamic imports for JS modules/components. +- **Font Optimization:** Use only the character sets you need. Subset fonts and use `font-display: swap`. + +### Network Optimization +- **Reduce HTTP Requests:** Combine files, use image sprites, and inline critical CSS. +- **HTTP/2 and HTTP/3:** Enable these protocols for multiplexing and lower latency. +- **Client-Side Caching:** Use Service Workers, IndexedDB, and localStorage for offline and repeat visits. +- **CDNs:** Serve static assets from a CDN close to your users. Use multiple CDNs for redundancy. +- **Defer/Async Scripts:** Use `defer` or `async` for non-critical JS to avoid blocking rendering. +- **Preload and Prefetch:** Use `` and `` for critical resources. + +### JavaScript Performance +- **Avoid Blocking the Main Thread:** Offload heavy computation to Web Workers. +- **Debounce/Throttle Events:** For scroll, resize, and input events, use debounce/throttle to limit handler frequency. +- **Memory Leaks:** Clean up event listeners, intervals, and DOM references. Use browser dev tools to check for detached nodes. +- **Efficient Data Structures:** Use Maps/Sets for lookups, TypedArrays for numeric data. +- **Avoid Global Variables:** Globals can cause memory leaks and unpredictable performance. +- **Avoid Deep Object Cloning:** Use shallow copies or libraries like lodash's `cloneDeep` only when necessary. + +### Accessibility and Performance +- **Accessible Components:** Ensure ARIA updates are not excessive. Use semantic HTML for both accessibility and performance. +- **Screen Reader Performance:** Avoid rapid DOM updates that can overwhelm assistive tech. + +### Framework-Specific Tips +#### React +- Use `React.memo`, `useMemo`, and `useCallback` to avoid unnecessary renders. +- Split large components and use code-splitting (`React.lazy`, `Suspense`). +- Avoid anonymous functions in render; they create new references on every render. +- Use `ErrorBoundary` to catch and handle errors gracefully. +- Profile with React DevTools Profiler. + +#### Angular +- Use OnPush change detection for components that don't need frequent updates. +- Avoid complex expressions in templates; move logic to the component class. +- Use `trackBy` in `ngFor` for efficient list rendering. +- Lazy load modules and components with the Angular Router. +- Profile with Angular DevTools. + +#### Vue +- Use computed properties over methods in templates for caching. +- Use `v-show` vs `v-if` appropriately (`v-show` is better for toggling visibility frequently). +- Lazy load components and routes with Vue Router. +- Profile with Vue Devtools. + +### Common Frontend Pitfalls +- Loading large JS bundles on initial page load. +- Not compressing images or using outdated formats. +- Failing to clean up event listeners, causing memory leaks. +- Overusing third-party libraries for simple tasks. +- Ignoring mobile performance (test on real devices!). + +### Frontend Troubleshooting +- Use Chrome DevTools' Performance tab to record and analyze slow frames. +- Use Lighthouse to audit performance and get actionable suggestions. +- Use WebPageTest for real-world load testing. +- Monitor Core Web Vitals (LCP, FID, CLS) for user-centric metrics. + +--- + +## Backend Performance + +### Algorithm and Data Structure Optimization +- **Choose the Right Data Structure:** Arrays for sequential access, hash maps for fast lookups, trees for hierarchical data, etc. +- **Efficient Algorithms:** Use binary search, quicksort, or hash-based algorithms where appropriate. +- **Avoid O(n^2) or Worse:** Profile nested loops and recursive calls. Refactor to reduce complexity. +- **Batch Processing:** Process data in batches to reduce overhead (e.g., bulk database inserts). +- **Streaming:** Use streaming APIs for large data sets to avoid loading everything into memory. + +### Concurrency and Parallelism +- **Asynchronous I/O:** Use async/await, callbacks, or event loops to avoid blocking threads. +- **Thread/Worker Pools:** Use pools to manage concurrency and avoid resource exhaustion. +- **Avoid Race Conditions:** Use locks, semaphores, or atomic operations where needed. +- **Bulk Operations:** Batch network/database calls to reduce round trips. +- **Backpressure:** Implement backpressure in queues and pipelines to avoid overload. + +### Caching +- **Cache Expensive Computations:** Use in-memory caches (Redis, Memcached) for hot data. +- **Cache Invalidation:** Use time-based (TTL), event-based, or manual invalidation. Stale cache is worse than no cache. +- **Distributed Caching:** For multi-server setups, use distributed caches and be aware of consistency issues. +- **Cache Stampede Protection:** Use locks or request coalescing to prevent thundering herd problems. +- **Don't Cache Everything:** Some data is too volatile or sensitive to cache. + +### API and Network +- **Minimize Payloads:** Use JSON, compress responses (gzip, Brotli), and avoid sending unnecessary data. +- **Pagination:** Always paginate large result sets. Use cursors for real-time data. +- **Rate Limiting:** Protect APIs from abuse and overload. +- **Connection Pooling:** Reuse connections for databases and external services. +- **Protocol Choice:** Use HTTP/2, gRPC, or WebSockets for high-throughput, low-latency communication. + +### Logging and Monitoring +- **Minimize Logging in Hot Paths:** Excessive logging can slow down critical code. +- **Structured Logging:** Use JSON or key-value logs for easier parsing and analysis. +- **Monitor Everything:** Latency, throughput, error rates, resource usage. Use Prometheus, Grafana, Datadog, or similar. +- **Alerting:** Set up alerts for performance regressions and resource exhaustion. + +### Language/Framework-Specific Tips +#### Node.js +- Use asynchronous APIs; avoid blocking the event loop (e.g., never use `fs.readFileSync` in production). +- Use clustering or worker threads for CPU-bound tasks. +- Limit concurrent open connections to avoid resource exhaustion. +- Use streams for large file or network data processing. +- Profile with `clinic.js`, `node --inspect`, or Chrome DevTools. + +#### Python +- Use built-in data structures (`dict`, `set`, `deque`) for speed. +- Profile with `cProfile`, `line_profiler`, or `Py-Spy`. +- Use `multiprocessing` or `asyncio` for parallelism. +- Avoid GIL bottlenecks in CPU-bound code; use C extensions or subprocesses. +- Use `lru_cache` for memoization. + +#### Java +- Use efficient collections (`ArrayList`, `HashMap`, etc.). +- Profile with VisualVM, JProfiler, or YourKit. +- Use thread pools (`Executors`) for concurrency. +- Tune JVM options for heap and garbage collection (`-Xmx`, `-Xms`, `-XX:+UseG1GC`). +- Use `CompletableFuture` for async programming. + +#### .NET +- Use `async/await` for I/O-bound operations. +- Use `Span` and `Memory` for efficient memory access. +- Profile with dotTrace, Visual Studio Profiler, or PerfView. +- Pool objects and connections where appropriate. +- Use `IAsyncEnumerable` for streaming data. + +### Common Backend Pitfalls +- Synchronous/blocking I/O in web servers. +- Not using connection pooling for databases. +- Over-caching or caching sensitive/volatile data. +- Ignoring error handling in async code. +- Not monitoring or alerting on performance regressions. + +### Backend Troubleshooting +- Use flame graphs to visualize CPU usage. +- Use distributed tracing (OpenTelemetry, Jaeger, Zipkin) to track request latency across services. +- Use heap dumps and memory profilers to find leaks. +- Log slow queries and API calls for analysis. + +--- + +## Database Performance + +### Query Optimization +- **Indexes:** Use indexes on columns that are frequently queried, filtered, or joined. Monitor index usage and drop unused indexes. +- **Avoid SELECT *:** Select only the columns you need. Reduces I/O and memory usage. +- **Parameterized Queries:** Prevent SQL injection and improve plan caching. +- **Query Plans:** Analyze and optimize query execution plans. Use `EXPLAIN` in SQL databases. +- **Avoid N+1 Queries:** Use joins or batch queries to avoid repeated queries in loops. +- **Limit Result Sets:** Use `LIMIT`/`OFFSET` or cursors for large tables. + +### Schema Design +- **Normalization:** Normalize to reduce redundancy, but denormalize for read-heavy workloads if needed. +- **Data Types:** Use the most efficient data types and set appropriate constraints. +- **Partitioning:** Partition large tables for scalability and manageability. +- **Archiving:** Regularly archive or purge old data to keep tables small and fast. +- **Foreign Keys:** Use them for data integrity, but be aware of performance trade-offs in high-write scenarios. + +### Transactions +- **Short Transactions:** Keep transactions as short as possible to reduce lock contention. +- **Isolation Levels:** Use the lowest isolation level that meets your consistency needs. +- **Avoid Long-Running Transactions:** They can block other operations and increase deadlocks. + +### Caching and Replication +- **Read Replicas:** Use for scaling read-heavy workloads. Monitor replication lag. +- **Cache Query Results:** Use Redis or Memcached for frequently accessed queries. +- **Write-Through/Write-Behind:** Choose the right strategy for your consistency needs. +- **Sharding:** Distribute data across multiple servers for scalability. + +### NoSQL Databases +- **Design for Access Patterns:** Model your data for the queries you need. +- **Avoid Hot Partitions:** Distribute writes/reads evenly. +- **Unbounded Growth:** Watch for unbounded arrays or documents. +- **Sharding and Replication:** Use for scalability and availability. +- **Consistency Models:** Understand eventual vs strong consistency and choose appropriately. + +### Common Database Pitfalls +- Missing or unused indexes. +- SELECT * in production queries. +- Not monitoring slow queries. +- Ignoring replication lag. +- Not archiving old data. + +### Database Troubleshooting +- Use slow query logs to identify bottlenecks. +- Use `EXPLAIN` to analyze query plans. +- Monitor cache hit/miss ratios. +- Use database-specific monitoring tools (pg_stat_statements, MySQL Performance Schema). + +--- + +## Code Review Checklist for Performance + +- [ ] Are there any obvious algorithmic inefficiencies (O(n^2) or worse)? +- [ ] Are data structures appropriate for their use? +- [ ] Are there unnecessary computations or repeated work? +- [ ] Is caching used where appropriate, and is invalidation handled correctly? +- [ ] Are database queries optimized, indexed, and free of N+1 issues? +- [ ] Are large payloads paginated, streamed, or chunked? +- [ ] Are there any memory leaks or unbounded resource usage? +- [ ] Are network requests minimized, batched, and retried on failure? +- [ ] Are assets optimized, compressed, and served efficiently? +- [ ] Are there any blocking operations in hot paths? +- [ ] Is logging in hot paths minimized and structured? +- [ ] Are performance-critical code paths documented and tested? +- [ ] Are there automated tests or benchmarks for performance-sensitive code? +- [ ] Are there alerts for performance regressions? +- [ ] Are there any anti-patterns (e.g., SELECT *, blocking I/O, global variables)? + +--- + +## Advanced Topics + +### Profiling and Benchmarking +- **Profilers:** Use language-specific profilers (Chrome DevTools, Py-Spy, VisualVM, dotTrace, etc.) to identify bottlenecks. +- **Microbenchmarks:** Write microbenchmarks for critical code paths. Use `benchmark.js`, `pytest-benchmark`, or JMH for Java. +- **A/B Testing:** Measure real-world impact of optimizations with A/B or canary releases. +- **Continuous Performance Testing:** Integrate performance tests into CI/CD. Use tools like k6, Gatling, or Locust. + +### Memory Management +- **Resource Cleanup:** Always release resources (files, sockets, DB connections) promptly. +- **Object Pooling:** Use for frequently created/destroyed objects (e.g., DB connections, threads). +- **Heap Monitoring:** Monitor heap usage and garbage collection. Tune GC settings for your workload. +- **Memory Leaks:** Use leak detection tools (Valgrind, LeakCanary, Chrome DevTools). + +### Scalability +- **Horizontal Scaling:** Design stateless services, use sharding/partitioning, and load balancers. +- **Auto-Scaling:** Use cloud auto-scaling groups and set sensible thresholds. +- **Bottleneck Analysis:** Identify and address single points of failure. +- **Distributed Systems:** Use idempotent operations, retries, and circuit breakers. + +### Security and Performance +- **Efficient Crypto:** Use hardware-accelerated and well-maintained cryptographic libraries. +- **Validation:** Validate inputs efficiently; avoid regexes in hot paths. +- **Rate Limiting:** Protect against DoS without harming legitimate users. + +### Mobile Performance +- **Startup Time:** Lazy load features, defer heavy work, and minimize initial bundle size. +- **Image/Asset Optimization:** Use responsive images and compress assets for mobile bandwidth. +- **Efficient Storage:** Use SQLite, Realm, or platform-optimized storage. +- **Profiling:** Use Android Profiler, Instruments (iOS), or Firebase Performance Monitoring. + +### Cloud and Serverless +- **Cold Starts:** Minimize dependencies and keep functions warm. +- **Resource Allocation:** Tune memory/CPU for serverless functions. +- **Managed Services:** Use managed caching, queues, and DBs for scalability. +- **Cost Optimization:** Monitor and optimize for cloud cost as a performance metric. + +--- + +## Practical Examples + +### Example 1: Debouncing User Input in JavaScript +```javascript +// BAD: Triggers API call on every keystroke +input.addEventListener('input', (e) => { + fetch(`/search?q=${e.target.value}`); +}); + +// GOOD: Debounce API calls +let timeout; +input.addEventListener('input', (e) => { + clearTimeout(timeout); + timeout = setTimeout(() => { + fetch(`/search?q=${e.target.value}`); + }, 300); +}); +``` + +### Example 2: Efficient SQL Query +```sql +-- BAD: Selects all columns and does not use an index +SELECT * FROM users WHERE email = 'user@example.com'; + +-- GOOD: Selects only needed columns and uses an index +SELECT id, name FROM users WHERE email = 'user@example.com'; +``` + +### Example 3: Caching Expensive Computation in Python +```python +# BAD: Recomputes result every time +result = expensive_function(x) + +# GOOD: Cache result +from functools import lru_cache + +@lru_cache(maxsize=128) +def expensive_function(x): + ... +result = expensive_function(x) +``` + +### Example 4: Lazy Loading Images in HTML +```html + + + + + +``` + +### Example 5: Asynchronous I/O in Node.js +```javascript +// BAD: Blocking file read +const data = fs.readFileSync('file.txt'); + +// GOOD: Non-blocking file read +fs.readFile('file.txt', (err, data) => { + if (err) throw err; + // process data +}); +``` + +### Example 6: Profiling a Python Function +```python +import cProfile +import pstats + +def slow_function(): + ... + +cProfile.run('slow_function()', 'profile.stats') +p = pstats.Stats('profile.stats') +p.sort_stats('cumulative').print_stats(10) +``` + +### Example 7: Using Redis for Caching in Node.js +```javascript +const redis = require('redis'); +const client = redis.createClient(); + +function getCachedData(key, fetchFunction) { + return new Promise((resolve, reject) => { + client.get(key, (err, data) => { + if (data) return resolve(JSON.parse(data)); + fetchFunction().then(result => { + client.setex(key, 3600, JSON.stringify(result)); + resolve(result); + }); + }); + }); +} +``` + +--- + +## References and Further Reading +- [Google Web Fundamentals: Performance](https://web.dev/performance/) +- [MDN Web Docs: Performance](https://developer.mozilla.org/en-US/docs/Web/Performance) +- [OWASP: Performance Testing](https://owasp.org/www-project-performance-testing/) +- [Microsoft Performance Best Practices](https://learn.microsoft.com/en-us/azure/architecture/best-practices/performance) +- [PostgreSQL Performance Optimization](https://wiki.postgresql.org/wiki/Performance_Optimization) +- [MySQL Performance Tuning](https://dev.mysql.com/doc/refman/8.0/en/optimization.html) +- [Node.js Performance Best Practices](https://nodejs.org/en/docs/guides/simple-profiling/) +- [Python Performance Tips](https://docs.python.org/3/library/profile.html) +- [Java Performance Tuning](https://www.oracle.com/java/technologies/javase/performance.html) +- [.NET Performance Guide](https://learn.microsoft.com/en-us/dotnet/standard/performance/) +- [WebPageTest](https://www.webpagetest.org/) +- [Lighthouse](https://developers.google.com/web/tools/lighthouse) +- [Prometheus](https://prometheus.io/) +- [Grafana](https://grafana.com/) +- [k6 Load Testing](https://k6.io/) +- [Gatling](https://gatling.io/) +- [Locust](https://locust.io/) +- [OpenTelemetry](https://opentelemetry.io/) +- [Jaeger](https://www.jaegertracing.io/) +- [Zipkin](https://zipkin.io/) + +--- + +## Conclusion + +Performance optimization is an ongoing process. Always measure, profile, and iterate. Use these best practices, checklists, and troubleshooting tips to guide your development and code reviews for high-performance, scalable, and efficient software. If you have new tips or lessons learned, add them here—let's keep this guide growing! + +--- + +