Movie Analyzer

Report

Introduction

The motivation behind this project stems from the need to unravel the intricate correlations within the movie industry. With a rich dataset sourced from CrowdFlower - Data for Everyone, we aim to delve into the relationships between various movie parameters, including film length, IMDB rating, distributing studio, genre, and worldwide gross.

Project Context: In the ever-evolving landscape of the film industry, understanding the dynamics that drive a movie’s success is paramount. Filmmakers, producers, and distributors face constant challenges in making informed decisions about production, marketing strategies, and investments. This project seeks to provide valuable insights that can shape these decisions.

Problem Statement: The central problem addressed is the identification and analysis of correlations among key movie parameters. We aspire to uncover meaningful connections between elements such as film length, ratings, and financial performance to discern what factors contribute most significantly to a movie’s success.

Objectives:

1. Develop a Correlation Analysis Tool: A Shiny web application that enables users to explore and visualize correlations between two selected parameters interactively.

2. Derive Correlation Insights: Through visualizations such as scatter plots, correlation matrices, and trend lines, we aim to provide users with a comprehensive understanding of the relationships between different movie-related parameters.

3. Summarize Key Findings: A detailed summary report outlining major conclusions, significant correlations, and insights from the analysis, offering a holistic view of the project’s outcomes.

Main Conclusions: Upon completion, the project yielded valuable insights into the correlations influencing a movie’s success. The Correlation Analysis Tool and associated visualizations provided a nuanced understanding of the complex relationships between different movie-related parameters. These findings contribute not only to industry professionals’ decision-making processes but also to the broader understanding of movie trends, ratings, and financial performance over the specified 40-year period.

Justification of approach

Deliverables: We crafted a set of deliverables to address the complexities of the movie industry:

1. Correlation Analysis Tool: A Shiny web application empowering users to interactively explore and visualize correlations between chosen movie parameters. This tool enhances user engagement and facilitates a deeper understanding of the intricate relationships within the dataset.

2. Correlation Insights and Visualizations: Visual representations embedded within the Shiny app. These visualizations are designed to distill complex correlations into accessible insights, aiding users in comprehending the nuanced connections between various movie-related parameters.

3. Summary Report: A comprehensive document summarizing major findings, significant correlations, and insights derived from the analysis.

Intended Audience: The primary audience for our deliverables encompasses a spectrum of stakeholders within the film industry:

• Filmmakers and Producers: Seeking insights into factors influencing a movie’s success for strategic decision-making in production and marketing.

• Distributors: Wanting to understand the correlations between different parameters to optimize distribution strategies.

• Researchers and Analysts: Interested in exploring trends and correlations in the movie industry for academic or analytical purposes.

• Enthusiasts: Individuals intrigued by the dynamics of the film industry, seeking a comprehensive overview of movie trends over the specified 40-year period.

Meeting Audience Needs:

• Correlation Analysis Tool: This interactive tool caters to the diverse needs of industry professionals by allowing them to customize their exploration of correlations. Filmmakers and distributors can refine their strategies based on real-time insights, while researchers gain a dynamic platform for in-depth analysis.

• Correlation Insights and Visualizations: Visual representations make complex correlations accessible. Filmmakers and producers can quickly grasp the impact of parameters on a movie’s success, guiding decision-making. Researchers benefit from visually supported insights for academic exploration.

• Summary Report: The report condenses the project’s outcomes, providing a comprehensive resource for a broad audience. Filmmakers and distributors gain actionable insights, while researchers and enthusiasts access a distilled overview of the industry’s trends.

Design Process Summary

The design process for our deliverables involved a systematic and collaborative approach:

1. Requirement Analysis:

   • Key Challenges: Understanding the diverse needs of stakeholders within the film industry, ranging from filmmakers to researchers and think of designs that we can implement.

   • Considerations: Identifying the essential parameters for correlation analysis and ensuring the tool’s flexibility for customization.

2. Prototyping:

   • Key Challenges: Translating complex correlations into an intuitive and user-friendly interface.

   • Considerations: Prioritizing the clarity of visualizations, ensuring users can easily interpret and derive insights, checking other similar projects for ideas

3. Collaborative Development:

   • Key Challenges: Ensuring seamless integration of visualizations within the Shiny app and Git.

   • Considerations: Balancing functionality with a visually appealing and intuitive design for a positive user experience and documenting on Github

4. Testing and Refinement:

   • Key Challenges: Addressing potential technical glitches and refining visualizations based on user feedback.

   • Considerations: Conducting thorough testing to identify and rectify any issues, incorporating user feedback for continuous improvement.

5. Documentation:

   • Key Challenges: Summarizing findings in the report

   • Considerations: Crafting a report that presents key insights in a clear and accessible format, ensuring it serves both industry professionals and enthusiasts.

Data description

The dataset includes the following key attributes, making it well-suited for comprehensive analyses related to the movie industry:

1. Observations and Attributes:

• Observations (Rows): 437

• Attributes (Columns): 11

Main_Genre: The primary genre of the movie.

Genre_2: Additional information about the genre.

Genre_3: Further genre information; some entries are NA (Not Available).

imdb_rating: The IMDb rating of the movie.

length: The duration of the movie in minutes.

rank_in_year: The rank of the movie among the top movies in its respective year.

rating: The MPAA rating of the movie (e.g., PG-13, PG, R).

studio: The studio that produced/distributed the movie.

title: The title of the movie.

worldwide_gross: The worldwide gross box office receipts for the movie.

year: The year in which the movie was released

2. Creation of the Dataset

Purpose: The dataset was created to gather information about the ten most popular movies for each year over a span of 40 years (1975-2015).

Funding: The dataset was generated as a result of a data categorization project organized by CrowdFlower.

3. Influencing Factors on Data Collection: The data collection was influenced by a data categorization job, where the crowd was asked to find information about the most popular movies for each year. This likely influenced the inclusion of popular movies but might have excluded less well-known films.

4. Preprocessing and Data Form:

• Preprocessing: The data underwent categorization, likely involving tasks such as extracting movie titles, gathering genre information, obtaining run time, ratings, and box office receipts.

• Data Form: The data is organized in tabular form with rows representing individual movies and columns containing information about the movies.

5. Awareness and Purpose: It’s not explicitly mentioned whether the people involved were aware, but given it was a crowd-based task, participants likely knew they were contributing to a movie information dataset.The data was likely collected with the purpose of creating a comprehensive resource for analyzing movie industry trends, ratings, and financial performance over the specified 40-year period. Participants might have expected the data to be used for research and analysis in the field of movie studies or related domains.Design process

#|label: import data
library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.2     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.3     ✔ tibble    3.2.1
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

library(dplyr)

url <- "https://drive.google.com/uc?export=download&id=1psklLnp-K0A7ggoPexxykmWY7vB0DAds"

# Specify the column types
column_types <- cols(
  Main_Genre = col_character(),
  Genre_2 = col_character(),
  Genre_3 = col_character(),
  rating = col_character(),
  studio = col_character(),
  title = col_character(),
  worldwide_gross = col_character(),
  imdb_rating = col_double(),
  length = col_double(),
  rank_in_year = col_double(),
  year = col_double()
)

# Read the CSV file with specified column types
blockbusters <- read_csv(url, col_types = column_types)

#Minimal cleaning

# Convert "worldwide_gross" to numeric
blockbusters$worldwide_gross <- as.numeric(gsub("\\$", "", gsub(",", "", blockbusters$worldwide_gross)))

# Clean the "rating" column (assuming you want to keep only numeric part)
blockbusters$rating <- as.numeric(gsub(".*?([0-9.]+).*", "\\1", blockbusters$rating))

Warning: NAs introduced by coercion

# Optionally, convert "year" and "rank_in_year" to integers
blockbusters$year <- as.integer(blockbusters$year)
blockbusters$rank_in_year <- as.integer(blockbusters$rank_in_year)

library(shiny)
library(ggplot2)
library(DT)

Attaching package: 'DT'

The following objects are masked from 'package:shiny':

    dataTableOutput, renderDataTable

library(rsconnect)

Attaching package: 'rsconnect'

The following object is masked from 'package:shiny':

    serverInfo

ui <- fluidPage(
    titlePanel("Movie Analysis"),
    
    sidebarLayout(
        sidebarPanel(
            textInput("search", "Search for a movie:"),
            numericInput("startYear", "Start Year:", value = 1975, min = 1975, max = 2010),
            numericInput("endYear", "End Year:", value = 1975, min = 1975, max = 2010),
            selectInput("label1", "Choose a label:",
                        choices = c("NA", "Main_Genre", "length", "imdb_rating", "worldwide_gross"), selected = ""),
            selectInput("label2", "Choose a label:",
                        choices = c("NA", "Main_Genre", "length", "imdb_rating", "worldwide_gross"), selected = "")
        ),
        
        mainPanel(
            uiOutput("dynamicOutput")
        )
    )
)

server <- function(input, output) {
    
    filtered_data <- reactive({
        data <- blockbusters
        if (input$search != "") {
            data <- data[grepl(input$search, data$title, ignore.case = TRUE), ]
        }
        if (!is.null(input$startYear) && !is.null(input$endYear)) {
            data <- data[data$year >= input$startYear & data$year <= input$endYear, ]
        }
        data
    })

    # Reactive value to track if the plot has been generated
    plotGenerated <- reactiveVal(FALSE)

    observe({
        plotGenerated(input$label1 != "NA" && input$label2 != "NA" && input$label1 != input$label2 ||
                      input$label1 != "NA" && input$label2 == "NA")
    })

    output$dynamicOutput <- renderUI({
        if (plotGenerated()) {
            plotOutput("plotOutput")
        } else {
            DTOutput("searchResults")
        }
    })
    
    output$searchResults <- renderDT({
        filtered_data()
    }, options = list(pageLength = 5, autoWidth = TRUE))

    output$plotOutput <- renderPlot({
        data <- filtered_data()
        req(data)
        
       if (input$label1 != "NA" && input$label2 == "NA") {
        if (is.numeric(data[[input$label1]])) {
            # Numeric data: Use histogram
            ggplot(data, aes_string(x = input$label1)) +
                geom_histogram(bins = 30, fill = "skyblue2", color = "black") +
                theme_minimal() +
                labs(x = input$label1, y = "Frequency")
        } else {
            # Categorical data: Use bar plot
            ggplot(data, aes_string(x = input$label1)) +
                geom_bar(bins = 30, fill = "skyblue2", color = "black") +
                theme_minimal() +
                labs(x = input$label1, y = "Count")
        }
       }else if (input$label1 != "NA" && input$label2 != "NA" && input$label1 != input$label2) {
            # Two different labels are selected, show the correlation scatter plot
            ggplot(data, aes_string(x = input$label1, y = input$label2)) +
                geom_point(colour = "skyblue2",alpha = 0.5) +
                theme_minimal() +
                labs(x = input$label1, y = input$label2)
        }
    })
}

shinyApp(ui = ui, server = server)

Shiny applications not supported in static R Markdown documents

Limitations

Certain limitations in our project:

1. Sample Bias:

   • Limitation: Dataset bias towards popular movies.

   • Improvement: Include a more diverse movie selection to capture a broader industry spectrum.

2. Temporal Trends:

   • Limitation: Dataset may not fully represent recent industry changes.

   • Improvement: Regularly update the dataset to reflect current industry trends.

3. Rating System Biases:

   • Limitation: Ratings subject to biases, especially on platforms like IMDb.

   • Improvement: Implement sentiment analysis or incorporate diverse rating sources for a nuanced understanding.

4. Inflation Adjustment Challenges:

   • Limitation: Method for adjusting box office receipts for inflation may need refinement.

   • Improvement: Explore alternative inflation adjustment methods or collaborate with experts for accuracy.

5. User Feedback Limitations:

   • Limitation: Constraints on user feedback diversity and volume.

   • Improvement: Conduct more extensive user testing, engaging a broader range of stakeholders for comprehensive insights.

Opportunities for Improvement:

1. Dynamic Data Updates:

   • Improvement: Implement a mechanism for dynamic dataset updates to ensure ongoing relevance.

2. Machine Learning Integration:

   • Improvement: Explore integrating machine learning models for predictive analytics.

3. Enhanced Visualizations:

   • Improvement: Invest in advanced visualization techniques for clearer and impactful representations.

4. Incorporating External Factors:

   • Improvement: Expand the scope to include external factors influencing movie success.

5. User Training Resources:

   • Improvement: Develop tutorials to assist users in interpreting visualizations, especially those less familiar with data analysis.

Acknowledgments

We would like to acknowledge:

1. Course Instructor and Peers:

   • We express our gratitude to our course instructor for his guidance and insights throughout the learning process. Additionally, the collaborative spirit of our peers contributed to a supportive learning environment.

2. CrowdFlower - Data for Everyone:

   • The dataset used in this project was sourced from CrowdFlower, and we extend our gratitude to the contributors who participated in the data categorization job.

3. Tidyverse and dplyr Packages:

   • We appreciate the invaluable support provided by the Tidyverse and dplyr packages in R, streamlining data manipulation and analysis processes.

4. Shiny Web Application Framework:

   • Our thanks go to the creators of Shiny, the web application framework in R. It empowered us to develop an interactive Correlation Analysis Tool for the exploration of movie-related parameters.

5. Online Learning Resources:

   • Numerous online tutorials and learning resources played a pivotal role in enhancing our understanding of data analysis techniques and Shiny app development. These resources, including Stack Overflow questions and YouTube tutorials (https://www.youtube.com/watch?v=9uFQECk30kA) , were instrumental in overcoming challenges.

6. OpenAI - GPT-3.5 Model:

   • We have utilized OpenAI for helping us with debugging and code corrections.