member-import

Bulk Member Import

Import multiple members from CSV or Excel files into a Lobbi organization with comprehensive validation, duplicate detection, batch processing, and detailed error reporting.

Usage

/member-import <file-path> --org=<org-id> [options]

Description

This command imports member data in bulk by:

• Parsing CSV or Excel files with flexible column mapping
• Validating all member data against schema rules
• Detecting and handling duplicates
• Creating Keycloak users with proper authentication
• Inserting member records into MongoDB
• Processing in configurable batches for performance
• Generating comprehensive import reports
• Sending welcome emails to new members

Prerequisites

Required Services:

• Keycloak running and accessible
• MongoDB connection available
• Redis for job queue (optional, for async processing)
• SMTP configured for welcome emails

Environment Variables:

DATABASE_URL=mongodb://admin:admin@localhost:27017/member_db?authSource=admin
KEYCLOAK_URL=http://localhost:8080
KEYCLOAK_REALM=alpha-members
KEYCLOAK_CLIENT_ID=member-api
KEYCLOAK_CLIENT_SECRET=<secret>
SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_USER=apikey
SMTP_PASSWORD=<api-key>
REDIS_URL=redis://localhost:6379

File Format Requirements:

CSV/Excel files must include these columns (case-insensitive):

• Required: email, firstName, lastName
• Optional: phone, dateOfBirth, membershipType, status, joinDate, address, city, state, zipCode, country

Example CSV:

email,firstName,lastName,phone,membershipType,status
[email protected],John,Doe,555-1234,premium,active
[email protected],Jane,Smith,555-5678,basic,active

Step-by-Step Instructions

1. Validate Input File

# Check file exists
test -f <file-path> || { echo "ERROR: File not found"; exit 1; }

# Detect file type
FILE_EXT="${FILE_PATH##*.}"
case "$FILE_EXT" in
  csv) echo "CSV file detected" ;;
  xlsx|xls) echo "Excel file detected" ;;
  *) echo "ERROR: Unsupported file type. Use CSV or Excel"; exit 1 ;;
esac

# Preview first 5 rows
if [ "$FILE_EXT" = "csv" ]; then
  head -6 <file-path> | column -t -s,
else
  # Use xlsx2csv or similar for Excel
  python3 << 'EOF'
import pandas as pd
df = pd.read_excel("<file-path>", nrows=5)
print(df.to_string())
EOF
fi

# Count total rows
if [ "$FILE_EXT" = "csv" ]; then
  TOTAL_ROWS=$(($(wc -l < <file-path>) - 1))
else
  TOTAL_ROWS=$(python3 -c "import pandas as pd; print(len(pd.read_excel('<file-path>')))")
fi

echo "Total members to import: $TOTAL_ROWS"